-- Logs begin at Tue 2022-05-24 14:49:58 UTC, end at Tue 2022-05-24 16:00:10 UTC. -- May 24 15:23:12.519854 dv-cinder-843153-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 24 15:23:12.528723 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 15:23:12 2022] *** May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: nodename: dv-cinder-843153-1-iscsi May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: machine: x86_64 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: clock source: unix May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: pcre jit disabled May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: detected number of CPU cores: 4 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: current working directory: / May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: detected binary path: /usr/bin/uwsgi-core May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: your processes number limit is 47680 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: your memory page size is 4096 bytes May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: detected max file descriptor number: 2048 May 24 15:23:12.530776 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: lock engine: pthread robust mutexes May 24 15:23:12.531550 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: thunder lock: enabled May 24 15:23:12.531550 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 24 15:23:12.531550 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 15:23:12.575345 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: Python main interpreter initialized at 0x556b8d4c59c0 May 24 15:23:12.575345 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: python threads support enabled May 24 15:23:12.575345 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: your server socket listen backlog is limited to 100 connections May 24 15:23:12.575345 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: your mercy for graceful operations on workers is 90 seconds May 24 15:23:12.575682 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: mapped 403077 bytes (393 KB) for 2 cores May 24 15:23:12.575755 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: *** Operational MODE: preforking *** May 24 15:23:12.576023 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: *** uWSGI is running in multiple interpreter mode *** May 24 15:23:12.576023 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: spawned uWSGI master process (pid: 80229) May 24 15:23:12.576226 dv-cinder-843153-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 24 15:23:12.577285 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: spawned uWSGI worker 1 (pid: 80237, cores: 1) May 24 15:23:12.577285 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: spawned uWSGI worker 2 (pid: 80238, cores: 1) May 24 15:23:12.577285 dv-cinder-843153-1-iscsi devstack@keystone.service[80229]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 15:23:18.158336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:23:18.158336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes May 24 15:23:18.159181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:23:18.159181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes May 24 15:23:18.160459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:23:18.160459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes May 24 15:23:18.160984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:23:18.160984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes May 24 15:23:18.207650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 15:23:18.207836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 15:23:18.207969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 15:23:18.208292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 15:23:18.208439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 15:23:18.208717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.208957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.209126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 15:23:18.209229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.209309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.209374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 15:23:18.209446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.209507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] command line args: [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 15:23:18.209645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.209888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 15:23:18.209888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ================================================================================ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 15:23:18.210009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_dir = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_source = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.210980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] debug = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.211918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_config_append = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_dir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.212888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.213984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.214951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.215937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] publish_errors = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.216925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.217944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] standard_threads = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.218454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.218757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_journal = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.218893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.218964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_json = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.219110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.219247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_stderr = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.219247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_syslog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.219395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.219467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:23:18.219544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.219728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.219866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.219947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.220918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.221946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.222153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.222746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.222832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.222904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.222973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.223791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.224949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.225970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.226883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.227918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.228920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.229961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.230974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.231990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.232921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.233871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.234920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.235821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.236995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.237860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.238898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.239936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.240904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.241932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.242977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.243904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.external = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.244899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.password = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.245830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.token = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.246942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.247904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.248981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.249889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250483 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.250944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.251957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.252937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.253051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.253173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.253296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.253496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.254896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.255946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.256891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.257990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.258902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.259895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.260975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.261933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.262903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.263951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.264898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.265948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.266976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.267902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.268981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.269526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.269675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.269794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.269909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.270946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.271995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.272929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.273978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.274905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.275780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.276952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.277991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.278930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.279981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.280959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.281994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.282923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.283945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.284077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.284208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.284324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.284437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.284550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.285962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.286893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.287932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.288906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.289953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.290997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.291927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.292993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.293912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.294902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.295016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 15:23:18.295138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.295138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.295138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.295138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:23:18.295138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 15:23:18.322716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.322936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.323146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.323459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.323691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.323917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.331850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.332738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.360201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.360468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.361194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.361392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.412819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.413061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.464436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.464687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.465476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.465662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.530980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.531100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.531181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.531306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.531388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.531509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.554787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.554933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.555925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.556067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.556192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.556313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.556482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.567585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.567714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.580060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.580318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.580538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.580750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.580959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.581169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.581375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.582155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.582390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.582610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.582818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.583034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.583245 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.583450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:23:18.600300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.600502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:23:18.600825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.601016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:23:18.601128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.601268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:23:18.601410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.601492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.601617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.601698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:23:18.601827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.602028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.602153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.602286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.602458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.602723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.612992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.613290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.613611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.613735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.613886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.614075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.614164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.614431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.621941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.622457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.625212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.625487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.625690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.625791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.625978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.626940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.627896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.628020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.628175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.628513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.654393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.654730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.656132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.656413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.656606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.656702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.661944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.662071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.662278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.662360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.662441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.662518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.669782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.669919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.670059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.670152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.674728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.674876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.675120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.675313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.675470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.675593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.675816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.675946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.676985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.695306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.695434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.695557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.695661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.700176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.700305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.703744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.703873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.704051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.704136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.709082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.709210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.712737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.712951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.716492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.716692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.716819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.716957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.717899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.718046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.736729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.736856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.736995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.737095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.737213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.737293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.737498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.737662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.745515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.745706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.747200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.747425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.747552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.747691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.750500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.750709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.750835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.750970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751053 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.751766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.764980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.765108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.765263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.765351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.765514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.765663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.765803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.765885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.766673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.784495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.784666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.787789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.787918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.791082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.791250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.792724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.792854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.794321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.794450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.794589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.794670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.794876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.795003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.802600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.802728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.804202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.804347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.807397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.807525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.807664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.807746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.807945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.808108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.808195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.808271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.817264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.817392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.817550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.817634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.817903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.818030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.825690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.825838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:23:18.825978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.826877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.827895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.828940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.829060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.829165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:23:18.867576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556b8d4c59c0 pid: 80237 (default app) May 24 15:23:18.868071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556b8d4c59c0 pid: 80238 (default app) May 24 15:23:18.870840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669bb00-2968-410e-8390-a7a86e9bfb91 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:18.871101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669bb00-2968-410e-8390-a7a86e9bfb91 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:18.871394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669bb00-2968-410e-8390-a7a86e9bfb91 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:18.875711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Tue May 24 15:23:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:26.795206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:26.795820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:26.795820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:26.800629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:23:26.806992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:26.807753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:26.807891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:26.885432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: INFO dbcounter [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] Registered counter for database keystone May 24 15:23:26.907026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 15:23:26.917981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 15:23:27.102425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 15:23:27.102829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 15:23:27.159742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:27.161470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:27.291858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 15:23:27.499200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:27.569419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: INFO dbcounter [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] Registered counter for database keystone May 24 15:23:27.591050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG oslo_db.sqlalchemy.engines [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80238) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 15:23:27.603879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writer thread running {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 15:23:27.729557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693b612-6ad0-4515-a3b4-647c8c3ed244 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:27.735128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:27.735789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:27.736057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:27.736380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:27.745383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:27.797477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING keystone.common.rbac_enforcer.enforcer [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 15:23:27.798034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c35453-8b4e-4c04-a7ec-c5d89e4b1c42 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:27.808182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Tue May 24 15:23:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:27.815200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:27.839123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc5275-bb77-4c46-86cb-c02d266b5fd1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:27.848334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:27.849455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:27.849735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:27.850003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] PATH_INFO: `/v3/projects/9a64b5dceb3e44fc817c5b6566c89325` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:27.851661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] RBAC: Authorizing `identity:get_project(project_id=9a64b5dceb3e44fc817c5b6566c89325)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:28.012052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 15:23:28.012425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1142893-44b8-4e23-922c-396d53a46d46 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:28.017167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Tue May 24 15:23:27 2022] GET /identity/v3/projects/9a64b5dceb3e44fc817c5b6566c89325 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:30.166805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc66e6-d806-4794-8179-e8f94c6bbcde None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.167675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc66e6-d806-4794-8179-e8f94c6bbcde None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.167675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc66e6-d806-4794-8179-e8f94c6bbcde None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.174742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:30.181552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.181994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.182118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.205467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:30.207057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:30.215708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 15:23:30.238243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc66e6-d806-4794-8179-e8f94c6bbcde None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:30.254871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc66e6-d806-4794-8179-e8f94c6bbcde None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:30.259891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:30.260556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.260833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.261107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.269449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:30.271254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb703e-a831-4c34-ac6e-c36c286b36e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:30.280175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Tue May 24 15:23:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:30.285931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:30.295213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77beda-df31-4a52-befc-489c7cd6560b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:30.301617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:30.302502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.302783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.303053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.304629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:30.306490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97eded6-3add-4f8d-8936-620034b86cbb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:30.308068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Tue May 24 15:23:30 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:32.539739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a864-c625-4fc4-b2c4-8d3237ee7630 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.540312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a864-c625-4fc4-b2c4-8d3237ee7630 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.540312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a864-c625-4fc4-b2c4-8d3237ee7630 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.541700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:23:32.547847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.548316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.548433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.567205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:32.568415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:32.581587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146ffc-e6e1-4b4c-821e-70baf9fea856 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.581873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146ffc-e6e1-4b4c-821e-70baf9fea856 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.582153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49146ffc-e6e1-4b4c-821e-70baf9fea856 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.583535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:23:32.588727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.589017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.589298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.629437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80238) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 15:23:32.630121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80238) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 15:23:32.655237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:32.676592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:32.678177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:32.678398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.703385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:32.776971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:32.786999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe15aa-89ca-445e-9e10-adc4a553c28d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:32.799481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.825030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:32.825861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.826141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.826415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.838554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:32.849785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:32.850994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:32.851805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:23:32.856947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.905998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6451-f800-4b93-9e0f-5899be72df41 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:32.908357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:32.944177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:32.945079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.945358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.945652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.945773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:32.966596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4397-fd8c-4822-ae94-bdb819268e93 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.000418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.001085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.001369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.001641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.002392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:33.002695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.004218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.014120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.015662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.016067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:33.022394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.122270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:23:33.126563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49569a9-217f-4174-9e2c-9dabdacb0346 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.130382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.134332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:33.169264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.170155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.170431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.170711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.172341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.173836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.175496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:33.180934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.195339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:33.237577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7f38b-d53b-4121-b6d2-25c729c7727c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.270234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.271033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.271313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.271586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.281610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269eabd4-0a79-4f88-8468-446311510687 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.290059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.291575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:23:33.312800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:33.313623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:33.314404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:23:33.314404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:23:33.314404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:23:33.314404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:33.314404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 9/18] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:33.314792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.315775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.316133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.316605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.317799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.319346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.319954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.320981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/19] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:33.326320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.376315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:33.404857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:33.421135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbf157-95b5-43bc-a1c8-644ebe74f0d7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.423595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341e823-d124-4e2f-b114-0610c69123df None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.454210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.454896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.455178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.455328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.455448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.455977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.456221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:33.456382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.456520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.456634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.457539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.domain.id=default, user.id=108385be6e5b428a910af5b70d71b0c3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:33.457841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.457928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02804-b151-4953-a3cd-b08264c57601 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.459265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc6ce3-2b72-4264-9902-084a7c599676 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.466116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/role_assignments?user.id=108385be6e5b428a910af5b70d71b0c3&scope.domain.id=default&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 233 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:33.477744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:36.115817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200820e8-cfd3-4d1e-97f1-2aca38beb209 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.116437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200820e8-cfd3-4d1e-97f1-2aca38beb209 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.116437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200820e8-cfd3-4d1e-97f1-2aca38beb209 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.117898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:36.124140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.125147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.125646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.281972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:36.283504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:36.320571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a272485-4417-495b-8c4c-93e976230937 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.321207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a272485-4417-495b-8c4c-93e976230937 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.321682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a272485-4417-495b-8c4c-93e976230937 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.323244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/23] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:23:36.328197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76762c9-9d95-4789-b6ff-b569a4bf9fd9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.328696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76762c9-9d95-4789-b6ff-b569a4bf9fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.329171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76762c9-9d95-4789-b6ff-b569a4bf9fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.356348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b76762c9-9d95-4789-b6ff-b569a4bf9fd9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:36.357938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b76762c9-9d95-4789-b6ff-b569a4bf9fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:36.402504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 11/24] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:36.435927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:36.477639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/25] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:36.479418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6a20-800f-4e76-b0df-4d9564fab1cd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.479899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6a20-800f-4e76-b0df-4d9564fab1cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.480417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f6a20-800f-4e76-b0df-4d9564fab1cd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.481991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/26] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:36.487629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.488167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.488630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.494724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:36.520949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:36.522123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:36.577859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dda8a-aab0-4e6e-bf9b-eac7d2d33823 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:36.617461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:36.618266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.618544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.618812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.630408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:36.639552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:36.640441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:36.641187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:36.642341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 12/27] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:36.642341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:36.698719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/28] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:36.699598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:36.712669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:36.745667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b6622-ee49-4931-aa50-42954b9fee06 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:36.753905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:36.788042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:36.788860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.789203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.789636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.790490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:36.792174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:36.827166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f004ef5-d95a-4573-94c4-0842af03009d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:36.862469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:36.863076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.863356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.863670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.864589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:36.864901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:36.866485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108efcea-f904-47f3-86d3-a131c7ccd9e5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:36.875266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/29] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:36.881752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395ce2-1b9d-4bfe-9183-dec52fd8e4b2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:36.882038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395ce2-1b9d-4bfe-9183-dec52fd8e4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:36.882375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04395ce2-1b9d-4bfe-9183-dec52fd8e4b2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.883793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/30] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:36.884816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04395ce2-1b9d-4bfe-9183-dec52fd8e4b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:36.927635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 13/31] 10.222.0.55 () {60 vars in 1131 bytes} [Tue May 24 15:23:36 2022] POST /identity/v3/roles => generated 219 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:36.928799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:36.986673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.049211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04395ce2-1b9d-4bfe-9183-dec52fd8e4b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.089991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.090998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.091332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.091605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.104229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.106114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.115153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ab3a-943e-44a9-ac90-f3c71c9d819a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ff405fda-7457-4baf-bb52-b005bd5de2d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:23:37.129044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:37.130238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:37.131686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:23:37.131686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:23:37.131686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:23:37.131686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:37.131686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/32] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:37.135045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235411da-6244-435b-88d2-083b33207b3e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.135045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235411da-6244-435b-88d2-083b33207b3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.135045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235411da-6244-435b-88d2-083b33207b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.163810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-235411da-6244-435b-88d2-083b33207b3e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:37.166561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-235411da-6244-435b-88d2-083b33207b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:37.194682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.195723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.196006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.196308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.198173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.200167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.202107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 14/33] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:36 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:37.203004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.242781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.259748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/34] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:37.260662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235411da-6244-435b-88d2-083b33207b3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.322931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477321d6-427c-473a-ac2d-a0534d7fdd90 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.333899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.374829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.375632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.375911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.376385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.377500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:37.377992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.379859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.392765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 15/35] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:37.393979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.402188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235411da-6244-435b-88d2-083b33207b3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.437807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.438615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.438887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.439153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.440339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.441820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16463c-da19-4008-ba9f-e0a6caaa3c3b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.443591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/36] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:37.444774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe10dd-0950-4444-a58c-f63bb2fb92cd None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.445057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe10dd-0950-4444-a58c-f63bb2fb92cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.445326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe10dd-0950-4444-a58c-f63bb2fb92cd None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.446641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/37] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:37.451106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe10dd-0950-4444-a58c-f63bb2fb92cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.499265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.507747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c6507a-cb57-44ca-8c5d-5f92e3e34fa0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.546091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.546764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.547348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.547620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.548422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.549821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.563069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe10dd-0950-4444-a58c-f63bb2fb92cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.572070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 16/38] 10.222.0.55 () {60 vars in 1131 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/roles => generated 225 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:37.572070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.581674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.602865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.603760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.604243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.604612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.605423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.607049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d659d6-2c2c-448e-aaf5-0dcf661f3b08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.715012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19028097-39ca-45ac-bef7-6206270c2e26 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:37.750321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/39] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/projects => generated 301 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:37.751849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.752182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.752459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.757294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:37.758436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.758840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.759274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] PATH_INFO: `/v3/domains/default/users/108385be6e5b428a910af5b70d71b0c3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.762235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=108385be6e5b428a910af5b70d71b0c3, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:37.763994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bbcba-06a3-42ec-a798-9b0ebd639028 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:37.775387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:37.777377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:37.784688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 17/40] 10.222.0.55 () {58 vars in 1361 bytes} [Tue May 24 15:23:37 2022] PUT /identity/v3/domains/default/users/108385be6e5b428a910af5b70d71b0c3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:37.786383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e4684-72b3-434b-8006-d84769ff4fb3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.786670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e4684-72b3-434b-8006-d84769ff4fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.786991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e4684-72b3-434b-8006-d84769ff4fb3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.788879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 18/41] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:37.794797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:37.795085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:37.795353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:37.817004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:37.818940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:37.840495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:37.872952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/42] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:37.907335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:37.941690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 19/43] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:37.970523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:38.034918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:38.045505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9688ff7-13e8-4dd0-8da2-a1beb53fd9de None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:38.089528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:38.090763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:38.091310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:38.091591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:38.093327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:38.095198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:38.095832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37008d98-bb32-4621-ac37-1a6b2a6a77d7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:38.097088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/44] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:38.102558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:38.134729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:38.135390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:38.135675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:38.135948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:38.136741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:38.138145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e50ccf-85da-4b1f-b6d7-adc90a11ade0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:38.153368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 20/45] 10.222.0.55 () {60 vars in 1131 bytes} [Tue May 24 15:23:37 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:23:38.211713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e4165-7f55-4173-a09f-8ee599ae8de9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:38.251258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:38.251944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:38.252258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:38.252538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:38.253360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:38.254900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5e50c-d1a3-44ae-a0e8-6b4ad075cca1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:38.290815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/46] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:23:38 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:23:40.031571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6071e-ab7e-4dff-acad-79ca251ca6e4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.032114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6071e-ab7e-4dff-acad-79ca251ca6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.032201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6071e-ab7e-4dff-acad-79ca251ca6e4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.033542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 21/47] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:40.039443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.039887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.040136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.057685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:40.058866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:40.072796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.131364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/48] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:40.155368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6071e-ab7e-4dff-acad-79ca251ca6e4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.160434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.263660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6071e-ab7e-4dff-acad-79ca251ca6e4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.299194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.300581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.300880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.301162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.312045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:40.320928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:40.321995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:40.322927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 22/49] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:40.328723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.438979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e8f4f-98bc-4c32-9a68-2a49f8a4dbcd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.475293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.475981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.476290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.476562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.477336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:40.477642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.479066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868d1de-69de-4ac2-8b7c-836f10524284 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.488574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/50] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:40.494457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.525180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d01dc-724e-4d2f-8f41-153ed590e233 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.525472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d01dc-724e-4d2f-8f41-153ed590e233 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.525746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d01dc-724e-4d2f-8f41-153ed590e233 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.527143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/51] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:40.532426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.532714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.532988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.551175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:40.552352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:40.555661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.562223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.603222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ada5ae-4d95-468e-be9e-4bf3235008d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.627309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/52] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:40.637785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.638672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.638953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.639222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.640518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.641985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.643801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 23/53] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:40.649125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.653046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.751602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521045e-1596-4375-a70d-1b7e940e6ff8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.757805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3568330-a00e-4c54-8202-d34f1c8683f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.785628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.786550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.786676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.786945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.787208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.788502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.790063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.791584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.791718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/54] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:40.792404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.792688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.792962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.794134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.795580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.802081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 24/55] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:40.802208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.803066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:40.808484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:40.898201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8facf-06be-4974-b334-37cb6bfdd2e5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.902751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8eb777-ad53-417b-adb3-51fb3e7c41df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:40.935094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.935755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.936015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.936361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.937189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.domain.id=default, user.id=108385be6e5b428a910af5b70d71b0c3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:40.937516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.938958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73177ccb-3b16-4ad4-8f84-be4766afd965 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.942266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:40.942929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:40.943211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:40.943466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:40.944197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:40.945345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/56] 10.222.0.55 () {56 vars in 1322 bytes} [Tue May 24 15:23:40 2022] GET /identity/v3/role_assignments?user.id=108385be6e5b428a910af5b70d71b0c3&scope.domain.id=default&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:40.945660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa03af9-df37-409d-bf36-731624eb55ca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:40.981030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 25/57] 10.222.0.55 () {60 vars in 1140 bytes} [Tue May 24 15:23:40 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:23:43.190590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b760-c4a9-44b2-8be5-bba7b355af2c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:43.191526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b760-c4a9-44b2-8be5-bba7b355af2c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:43.191675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551b760-c4a9-44b2-8be5-bba7b355af2c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:43.194050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/58] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:43.202032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:43.202459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:43.202563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:43.221200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:43.222598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:43.270733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:43.292694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 26/59] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:43.317545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551b760-c4a9-44b2-8be5-bba7b355af2c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:43.401063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:43.435219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551b760-c4a9-44b2-8be5-bba7b355af2c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:43.470388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:43.471186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:43.471464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:43.471938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:43.473188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:43.474654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827c410-442b-4ac5-925b-2f2c0d0862df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:43.476313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/60] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:43.482125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:43.584469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ae205-adb2-4bd8-bca8-b8c5beefb1de None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:43.618934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:43.619590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:43.619865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:43.620159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:43.620864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:43.622314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cbeb3-6dc6-4fb1-a97d-1b2c2ac14b1c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:43.653365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 27/61] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:23:43 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:46.894324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f95c4-1ac4-4a7d-8244-8e1c5b0cf926 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:46.894909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f95c4-1ac4-4a7d-8244-8e1c5b0cf926 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:46.894909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276f95c4-1ac4-4a7d-8244-8e1c5b0cf926 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:46.896386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/62] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:46.902079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47972-b57c-4230-b5ec-229232ce482d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:46.902496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47972-b57c-4230-b5ec-229232ce482d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:46.902668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc47972-b57c-4230-b5ec-229232ce482d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:46.933062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1bc47972-b57c-4230-b5ec-229232ce482d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:46.934285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1bc47972-b57c-4230-b5ec-229232ce482d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:46.969690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.013579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 28/63] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:47.029287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafc24-5bbe-4b78-8176-28cef968273e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.029578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafc24-5bbe-4b78-8176-28cef968273e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.029844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafc24-5bbe-4b78-8176-28cef968273e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.031225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/64] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:47.036727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbafc24-5bbe-4b78-8176-28cef968273e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.038619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263ea60-90d6-4a26-bfed-59b0242d62bf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.038907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263ea60-90d6-4a26-bfed-59b0242d62bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.039185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263ea60-90d6-4a26-bfed-59b0242d62bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.073660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1263ea60-90d6-4a26-bfed-59b0242d62bf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:47.074891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1263ea60-90d6-4a26-bfed-59b0242d62bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:47.201404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 29/65] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:47.233347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbafc24-5bbe-4b78-8176-28cef968273e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.259636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cf3ef-4188-48b1-b59b-ccc0544decd7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.259959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cf3ef-4188-48b1-b59b-ccc0544decd7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.260240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cf3ef-4188-48b1-b59b-ccc0544decd7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.261719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 30/66] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:23:47.266766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.267284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.267758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.280305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.281511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.281511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.281774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.289566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.289933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:47.292128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:47.294205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:47.302667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:47.303557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/67] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:47.304949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.385525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 31/68] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:47.392866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.434849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.456260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501e291-11a6-46bb-8fe7-a1a8fc4032cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.497819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.498564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.498844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.499116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.512179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:47.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:47.521581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:47.522326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/69] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:47.527513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.528905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f4e76-da51-471b-b0c3-ddac7768ef8d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.575139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.575955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.576261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.576536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.577288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:47.577788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:47.579004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:47.587744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.597468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 32/70] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:47.597468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.634037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623d154-e998-40e1-8e1a-14d180bb6088 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.671306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.672571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.672855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.673127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.684482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ffa97bb9-137c-403f-b828-acde20cf1401 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:47.692593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:47.693581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:47.694463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/71] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:47.695356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e094c-7e85-4285-87d7-85edb8f3c7f9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.695480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e094c-7e85-4285-87d7-85edb8f3c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.695760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e094c-7e85-4285-87d7-85edb8f3c7f9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.697052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/72] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:47.697742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e094c-7e85-4285-87d7-85edb8f3c7f9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.702776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.714814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe01d5-cb80-4fa6-a3bf-a3e3c0b60040 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.752262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.752845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.753137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.753402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.754151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:47.754784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:47.755880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efda52d-dc2c-41e9-a440-38dfbbdb0f1e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:47.764078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 33/73] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:47.765313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5220-056b-4097-a02a-7644514854d8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.765605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5220-056b-4097-a02a-7644514854d8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.765889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a5220-056b-4097-a02a-7644514854d8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.767257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 34/74] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:47.767825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a5220-056b-4097-a02a-7644514854d8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.810291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e094c-7e85-4285-87d7-85edb8f3c7f9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.830305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.843961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.844872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.845154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.845428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.846597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:47.848100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac2974-033f-4206-b4e7-17fcc6302284 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:47.850053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/75] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:47.851638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4c09-3398-4d5d-9624-e0ad8b61e84b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.851638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4c09-3398-4d5d-9624-e0ad8b61e84b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.851807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb4c09-3398-4d5d-9624-e0ad8b61e84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.867691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14fb4c09-3398-4d5d-9624-e0ad8b61e84b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:47.868897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14fb4c09-3398-4d5d-9624-e0ad8b61e84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:47.874617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a5220-056b-4097-a02a-7644514854d8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:47.909418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:47.910077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.910354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.910625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.911523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:47.911831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:47.913248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:47.921530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 35/76] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:47.922695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:47.936656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/77] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:47.939782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:47.941564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:47.941847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:47.942163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:47.959190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:47.960619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:48.028849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b25dfb-1745-4454-9c79-c455f92b2c11 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.029042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/78] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:48.029737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.064123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.065015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.065294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.065567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.077553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.080571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.082053 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.084339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 36/79] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:47 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:48.085115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.131303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c9f52a-09ff-478f-a967-c9bb891e176c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.153462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.164663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.165319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.165602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.165885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.185669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae270d-8828-440c-999f-24b7ccdb09db None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.218729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.219611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.219894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.220190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.221401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.222854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.224963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 37/80] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:48.225589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.235766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.237301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.286775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.321271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a986b-7d0b-4e3f-b007-38ea4a464a23 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.339095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/81] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:48.339956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.355514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.356352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.356670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.356938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.364489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.367645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:48.375634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:48.376493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:48.377254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 38/82] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:48.379350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.437400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e7d1e-a3bd-49ef-9513-2c83ca12a29a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.470107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.470900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.471176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.471443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.472634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.474352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.476197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/83] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:48.476774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.476938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53fa225-e856-4040-a61f-4209810ca836 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.493701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.513701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.514351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.514629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.514900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.572741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d0d4b-e1b4-4cdf-bc42-a1f59fd192a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.578283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.581166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.583494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.605989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.606646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.606924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.607200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.672342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.674180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.680747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 39/84] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:48.681675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.694357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.777906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/85] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:48.778613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.779209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.785908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97f446-b338-4772-9b46-d3cc2da36ea9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.821368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.822230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.822513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.822836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.823964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.825461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.827172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 40/86] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:48.827909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.881465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68926e-c4f8-4eb9-9486-28458f22704c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.897233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:48.915190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.915865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.916177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.916465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.917190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:48.917490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.918887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.925670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d110-fe8a-4a3b-b3ec-1e3d86350467 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:48.928083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/87] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:48.928873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:48.959056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:48.959723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:48.960000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:48.960296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:48.961013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:48.962473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:48.984411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 41/88] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:23:48 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:48.985222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.016171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.033272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7959b5-5c88-49e4-81cf-fe78a41cc608 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.071472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.072385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.072665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.072992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.074201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.075692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.077485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/89] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.083350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.104792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf60369-c3cc-4824-9ffd-f906a39cdbb2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.136594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.145897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.146791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.147085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.147343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.148889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.150301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.152095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 42/90] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:48 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.152889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.204572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb676bdc-43ef-41fe-a175-2739ce421dd6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.248589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.249262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.249566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.249817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.250653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=108385be6e5b428a910af5b70d71b0c3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:49.250964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.252452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.259957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/91] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/role_assignments?user.id=108385be6e5b428a910af5b70d71b0c3&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.260666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.268303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.281507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de05b76c-b9a2-49e2-ba3a-2a1b03f723da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.326507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.327333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.327614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.327883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.330199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.331782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.333779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 43/92] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.334597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.399513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b99c6f8-8889-468b-bb7d-f318ca7d839e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.411933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.452176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.453701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.454191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.454701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.456021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:49.457049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.459395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.466222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726995e3-71c6-4550-a1af-f0d6b556546d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.468332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/93] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.470482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.506613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.507298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.507577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.507848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.508744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:49.509172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.510808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.519791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 44/94] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.605258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f887b-ba49-4a2e-9109-94051a7c67c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.646400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.647483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.647961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.648457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.700301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.730288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.731965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.837756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/95] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:49.843638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:49.870994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.951193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f55ed20-0a52-4bf8-9443-b97a33e1a5da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:49.986379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:49.987270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.987551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.987828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] PATH_INFO: `/v3/projects/72bae80a77264968a3d4dc1212893568` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.989117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] RBAC: Authorizing `identity:get_project(project_id=72bae80a77264968a3d4dc1212893568)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:49.990673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f053e01-caa7-4bf8-80e7-1013e6a085cd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:49.992583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 45/96] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/projects/72bae80a77264968a3d4dc1212893568 => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:49.998714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:50.082663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:50.109363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5a1d0-4f55-4efd-87aa-6d1816782dad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:50.153234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:50.153819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:50.154101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:50.154372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:50.155206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=72bae80a77264968a3d4dc1212893568, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:50.155518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:50.157022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25363c0-c7c6-4d5e-a27e-1af76af4b8cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:50.168548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/97] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:49 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=72bae80a77264968a3d4dc1212893568&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:52.463870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9ccb-0085-41fb-a987-73913b9ec217 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.464528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9ccb-0085-41fb-a987-73913b9ec217 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.464528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a9ccb-0085-41fb-a987-73913b9ec217 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.466134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 46/98] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:52.481888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72a1d-4954-4fe3-9fe5-74db2ff81760 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.482432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72a1d-4954-4fe3-9fe5-74db2ff81760 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.482432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b72a1d-4954-4fe3-9fe5-74db2ff81760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.504300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68b72a1d-4954-4fe3-9fe5-74db2ff81760 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:52.505595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68b72a1d-4954-4fe3-9fe5-74db2ff81760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:52.608682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/99] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:52.649270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9ccb-0085-41fb-a987-73913b9ec217 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:52.753509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:52.763771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a9ccb-0085-41fb-a987-73913b9ec217 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:52.799994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:52.800831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.801151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.801801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.812259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:52.820263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:52.821248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:52.822067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 47/100] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:52.828189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a05eb-4878-4793-98b7-08194a235dc6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.828484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a05eb-4878-4793-98b7-08194a235dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.828768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a05eb-4878-4793-98b7-08194a235dc6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.830158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/101] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:52.831130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:52.837385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e0059-1460-4435-b8ae-0a99b807d0bf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.837667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e0059-1460-4435-b8ae-0a99b807d0bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.837950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e0059-1460-4435-b8ae-0a99b807d0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.863256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-505e0059-1460-4435-b8ae-0a99b807d0bf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:52.864990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-505e0059-1460-4435-b8ae-0a99b807d0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:52.940650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:52.947724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/102] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:52.950182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0092-e4ef-48ca-9f0d-1f36eb753d8d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.950690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0092-e4ef-48ca-9f0d-1f36eb753d8d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.951615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f0092-e4ef-48ca-9f0d-1f36eb753d8d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.953939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/103] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:52.956105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3506-1062-4901-b045-860ad1b027fa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.956915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3506-1062-4901-b045-860ad1b027fa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.957472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0e3506-1062-4901-b045-860ad1b027fa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.959618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/104] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:52.960921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:52.961411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:52.961889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:52.984621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:52.986628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:53.021898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83216172-2a53-45ec-9638-91963d3a8690 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.068248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.068951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.069234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.069813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.070731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:53.071155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.072830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec6845-5eff-4dfc-b34a-2abe5c6316c9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.078024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/105] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:53.079711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.087149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 48/106] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.087149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.087149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.087149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.106687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:53.108055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:53.123336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.179072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 49/107] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:53.180450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.215273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40d845-47f8-4664-b130-316dd0764cbb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.242752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.251131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.251988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.252292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.252563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.262487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:53.270701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:53.271663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:53.272785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/108] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:53.273422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.286879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312d666-403d-4411-81d3-a0fa0a456e2e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.321788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.322678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.322956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.323224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.324447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.325919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7ef6d-cff5-43cc-a17e-acde08f4e46b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.327588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 50/109] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.328761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4138bf-28d7-4ced-b0b3-fcaaeed1a29a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.329012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4138bf-28d7-4ced-b0b3-fcaaeed1a29a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.329287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4138bf-28d7-4ced-b0b3-fcaaeed1a29a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.330525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 51/110] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:53.331181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4138bf-28d7-4ced-b0b3-fcaaeed1a29a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.345331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.377723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc4f93-f38c-495f-a965-9d8028948498 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.411523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.412393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.412681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.412961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.414476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.415966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710f041-4d51-4bed-a7c6-85a794e962a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.417657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/111] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.418809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71d78-53bf-4486-99ea-15645f8f5610 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.419098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71d78-53bf-4486-99ea-15645f8f5610 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.419670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71d78-53bf-4486-99ea-15645f8f5610 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.420876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/112] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:53.421908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71d78-53bf-4486-99ea-15645f8f5610 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.435571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4138bf-28d7-4ced-b0b3-fcaaeed1a29a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.468457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.469261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.469550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.469826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.478637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.480101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:53.487563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:53.488702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 52/113] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:53.489813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.530469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71d78-53bf-4486-99ea-15645f8f5610 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.558429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.565383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.566051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.566327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.566648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.567395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:53.567691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.569124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd463-33bc-4a1e-9557-00a26ff308a7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.577035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/114] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.581330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffa1-31b5-45c1-8e8f-2b5f467c3192 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.581694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffa1-31b5-45c1-8e8f-2b5f467c3192 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.581967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05ffa1-31b5-45c1-8e8f-2b5f467c3192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.588652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8aa7a-2308-4001-81d6-903a55c23503 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.597517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed05ffa1-31b5-45c1-8e8f-2b5f467c3192 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:53.598754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed05ffa1-31b5-45c1-8e8f-2b5f467c3192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:53.622531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.623649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.623918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.624221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.625440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.626903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.628649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 53/115] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.629552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.669906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/116] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:53.684279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.684279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.684279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.691135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:53.692815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:53.730440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222e534-0de0-4f86-84f9-6b2d1c9f8302 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.736156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.758992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/117] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:53.760017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.776256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.776934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.777216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.777498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.778222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.779683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.790276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:53.806383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 54/118] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:23:53 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:53.807252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.870761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46be164-c0c8-4735-81aa-83de0cb44b0b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.909707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.910505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.910787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.911054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.912115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:53.912417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.913842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.928693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/119] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.928908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6d448-e633-4b85-9b90-5915b212f1d3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:53.928994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.964517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:53.965412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:53.965691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:53.965967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:53.967239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:53.968768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:53.970478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 55/120] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:53.970478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:53.975830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.042728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8fc1d6-bcd8-4c59-b542-637b052acb1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.043137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.081269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.082369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.082745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.083318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/108385be6e5b428a910af5b70d71b0c3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.085305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user_id=108385be6e5b428a910af5b70d71b0c3, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.086205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce06a9-f142-439b-8afb-7b1aabb9402e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.086912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.102212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/121] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:23:53 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/108385be6e5b428a910af5b70d71b0c3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:54.103024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.123391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.124358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.124758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.125139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.139961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:54.150938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:54.152094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:54.153042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:54.155806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 56/122] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:54.157161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.249533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.255250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa5034-78f9-4af8-a210-cf9839420a40 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.300823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92f7ac-54a1-421c-a4cc-80b576cb834c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.307562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.308444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.308728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.309004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.324342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:54.326385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:54.335540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:54.336537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:54.337344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:54.338505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/123] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:54.338505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.342569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.343465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.343748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.344021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] PATH_INFO: `/v3/users/dac17cc0cb2947d29561d6c51e66f189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.367607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] RBAC: Authorizing `identity:get_user(user_id=dac17cc0cb2947d29561d6c51e66f189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.369889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.372924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 57/124] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/users/dac17cc0cb2947d29561d6c51e66f189 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:54.373908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.464066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87887fe7-8d0c-4fb6-bc50-99b090fc9eca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.490132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864463-1c75-4e5e-931f-b1029fd7eceb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.507456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.508782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.509141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.509421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.510632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.512596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.514761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/125] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:54.515924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.526886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.527559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.527835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.528132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.528876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:54.529182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.530543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.539898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 58/126] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:54.540584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.549729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.590593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.642960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863f099-0939-401e-83d7-c3a3b7789053 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.653933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad91dc-c64b-4ac4-a0bb-c7b7061fafd7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.695153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.696079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.696509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.696932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.697819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:54.698266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.699006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.699807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.700166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.700270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.700653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.711016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/127] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:54.712018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.780247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.782117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.824806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.827178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:54.839467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7887f96-c630-4c24-aab9-65c881924896 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:54.881206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:54.882232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:54.882604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:54.883054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/d4c637fab90949129383c9f979fffa9f/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:54.885522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user_id=d4c637fab90949129383c9f979fffa9f, role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:54.887290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:54.894687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 59/128] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:54.895750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:54.911643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/129] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:23:54 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/d4c637fab90949129383c9f979fffa9f/roles/85a7aff706174bb09e06d1ca1d363973 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:54.912856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.003216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1070a-219e-4530-bf16-edfd7d95c6f6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.031478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884fa3cd-a810-41b4-93b6-a85421e81f64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.053448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.054621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.055026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.055450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.060887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.062610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.064515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 60/130] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.065428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.083233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:55.083429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.083429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.083429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.083429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.083734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.086061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.088092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/131] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:54 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.089070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.111811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:55.204786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2337b2f-1e14-40db-baf4-8ef08b76754a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.240670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.241627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.241905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.242209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.243632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.245136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.247354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/132] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:55 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.248144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.252476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73a006-f6bf-4962-82e7-0f2968b127f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.289224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.290193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.290540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.291035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.292450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.294086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.295972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 61/133] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:55 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.297185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.347552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:55.365303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aafdf-c543-46be-bb64-30512d470f90 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.402099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:55.406805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.408390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.408766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.409281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] PATH_INFO: `/v3/projects/72bae80a77264968a3d4dc1212893568` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.410566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] RBAC: Authorizing `identity:get_project(project_id=72bae80a77264968a3d4dc1212893568)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.412305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.414009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/134] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:55 2022] GET /identity/v3/projects/72bae80a77264968a3d4dc1212893568 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.414772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.447462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89c4d4-edf9-42a0-911f-689bb5da0158 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.523723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.524578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.524855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.525128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/d4c637fab90949129383c9f979fffa9f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.528242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user_id=d4c637fab90949129383c9f979fffa9f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.530900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.557857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 62/135] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:23:55 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/d4c637fab90949129383c9f979fffa9f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:55.558728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:55.591843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e75ea-8250-4b20-a514-08f25f318aba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.667039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.667039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.667039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.667039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.667692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=dac17cc0cb2947d29561d6c51e66f189) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:55.668064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.673507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e30f49-63de-44b1-b657-f90a1a9896f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.686759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/136] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:55 2022] GET /identity/v3/role_assignments?user.id=dac17cc0cb2947d29561d6c51e66f189&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:55.742228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcd8c0-b7a4-4ccf-abdd-3166b2de997d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:55.791657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:55.792969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:55.793525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:55.793980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] PATH_INFO: `/v3/projects/72bae80a77264968a3d4dc1212893568/users/d4c637fab90949129383c9f979fffa9f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:55.796982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] RBAC: Authorizing `identity:create_grant(project_id=72bae80a77264968a3d4dc1212893568, user_id=d4c637fab90949129383c9f979fffa9f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:55.798711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade6e32-044b-4748-896d-5f27539cf617 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:55.802839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:55.821611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 63/137] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:23:55 2022] PUT /identity/v3/projects/72bae80a77264968a3d4dc1212893568/users/d4c637fab90949129383c9f979fffa9f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:56.652856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ef7d-5045-4926-90c0-71faca918196 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:56.655423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ef7d-5045-4926-90c0-71faca918196 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:56.655423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627ef7d-5045-4926-90c0-71faca918196 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:56.655423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/138] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:56.662679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa78f3-89b8-4d7e-bfb3-6c1be2310e90 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:56.663134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa78f3-89b8-4d7e-bfb3-6c1be2310e90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:56.663322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa78f3-89b8-4d7e-bfb3-6c1be2310e90 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:56.664845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 64/139] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:56.676992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:56.677289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:56.677568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:56.694439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:56.694439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:56.694439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:56.721273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:56.723820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:56.741212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:56.742434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:56.756953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:56.868362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/140] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:56.877820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 65/141] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:56.902678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:56.911746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:57.052158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7555171-b89c-469b-b642-0124dfa81647 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.129867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.130697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.130979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.131254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:57.133757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:57.135890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:57.137744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/142] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:23:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:57.143149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:57.165060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3237951-a4d2-4233-bf82-4aa39f8c7439 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.180385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:57.215849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:57.231703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.233312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.233787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.234247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:57.255469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:57.264168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:57.265004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:23:57.265835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:57.274302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 66/143] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:23:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:57.280310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:57.390267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183633e-01f7-41f4-a631-e2ff4af08f97 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.462637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.463326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.463603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.463906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:57.465254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:57.466091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:57.467583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e5f7f-8b96-45fd-b3b9-88046486a921 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.513425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/144] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:23:57 2022] POST /identity/v3/users => generated 283 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:57.526087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.527125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.527622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.528115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:57.529062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:57.529574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:57.531169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:57.551654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 67/145] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:57.557392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:57.660739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:57.716086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fa43-80f9-49d2-9ee5-7ae7cb857cd2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.758885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.759845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.760154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.760428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] PATH_INFO: `/v3/users/108385be6e5b428a910af5b70d71b0c3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:57.764942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] RBAC: Authorizing `identity:get_user(user_id=108385be6e5b428a910af5b70d71b0c3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:57.766724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:57.768631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/146] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:57 2022] GET /identity/v3/users/108385be6e5b428a910af5b70d71b0c3 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:57.775285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:57.852399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:57.913433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb8949-4525-4bdf-b5a5-4aadb0d0aa80 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:57.949665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:57.950706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:57.951181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:57.951646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.023262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:58.025119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:58.086263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:58.141753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 68/147] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:58.149136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:58.292210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a2d8-9e1a-4a11-a2e7-2c32d3ecfab5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:58.333417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:58.334277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.334549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.334818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.335996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:58.337497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bb16f-f04f-4cf3-aec3-65449501386d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:58.339407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/148] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:23:58 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:58.345091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:58.435650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e3a-5af1-4457-b014-371daa8525a7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.435962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e3a-5af1-4457-b014-371daa8525a7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.436242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62435e3a-5af1-4457-b014-371daa8525a7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.438079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/149] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:58.443360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.443513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.443755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.469397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:58.470819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:58.474455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:58.488866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9424b5-0835-42b3-9483-8b71bb858a49 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:58.558592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:58.561435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.561963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.562437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.563497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=108385be6e5b428a910af5b70d71b0c3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:58.565124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:58.567810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:58.583737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 69/150] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:23:58 2022] GET /identity/v3/role_assignments?user.id=108385be6e5b428a910af5b70d71b0c3&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:58.605809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/151] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:23:58.644416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:58.668068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:58.801044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8f32-394c-4181-b165-f9cc56ada43c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:58.860993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:58.862081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.862494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.862894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.879742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f54981a4-9665-485d-8851-d9d45fb36e56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:58.888272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:58.889349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:23:58.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:58.892490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 70/152] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:23:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:58.898725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:58.977082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bfc0-42d6-428f-9404-1f491134dcbd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.977714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bfc0-42d6-428f-9404-1f491134dcbd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.978198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56bfc0-42d6-428f-9404-1f491134dcbd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:58.983687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 71/153] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:58.985377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f3a58-0c07-432a-ae4e-227845c4a699 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:58.985865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f3a58-0c07-432a-ae4e-227845c4a699 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:58.986325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f3a58-0c07-432a-ae4e-227845c4a699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.025083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f76f3a58-0c07-432a-ae4e-227845c4a699 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:59.026635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f76f3a58-0c07-432a-ae4e-227845c4a699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:59.032889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f504b0d-4b7a-456b-8303-74930d6783d7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.077815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:59.078500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.078774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.079045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.080017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:59.080349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:59.082751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7bd9b1-7fc0-4032-bde4-b94550d0aa27 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:59.089129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:59.094591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/154] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:23:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:59.096321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9d08-7842-424f-85ca-3014e2b6a1c9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.098855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9d08-7842-424f-85ca-3014e2b6a1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.098855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7e9d08-7842-424f-85ca-3014e2b6a1c9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.102174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/155] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:59.102174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e9d08-7842-424f-85ca-3014e2b6a1c9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:59.129556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 72/156] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:59.131187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.131655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.132731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.139231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:59.154172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:59.156523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:59.252622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 73/157] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:23:59.253414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:59.256711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7e9d08-7842-424f-85ca-3014e2b6a1c9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.303988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:59.304940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.305220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.305495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.307844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:59.309725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:59.311710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/158] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:23:59 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:59.312516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:59.443223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303471a6-85c3-4272-aef6-2c946a905ec5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.486341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:59.487314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:59.488383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.489376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.489872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.501724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:59.505545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:59.512801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:59.513800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:23:59.514542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 74/159] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:59.515887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:59.588267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda86b09-4cd9-44ce-9c20-59607b6ea1c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.658885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:59.661849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.662427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.669104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.698009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ffe1d98d-0878-42dc-a2fb-7a0538fe88f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:23:59.712581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:23:59.713559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:23:59.714335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:23:59.719494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cc525-302f-4b67-8425-c827e0c9e520 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.722721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/160] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:23:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:23:59.728614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d998b53-686c-49a7-a28a-d83dca638e73 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.729134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d998b53-686c-49a7-a28a-d83dca638e73 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.730489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d998b53-686c-49a7-a28a-d83dca638e73 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.732296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/161] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:23:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:23:59.733201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d998b53-686c-49a7-a28a-d83dca638e73 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:59.794997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:59.795654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.795932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.796960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:59.875690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:59.887909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:59.887909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1e8a-31c7-465b-8a5e-e790b3e0fdcb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:59.969367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d998b53-686c-49a7-a28a-d83dca638e73 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:59.991688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 75/162] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:23:59.993098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:59.993392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:59.993722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.022631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:00.023828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:00.026716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.042308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.043044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.043324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.043598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.044419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:00.044725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.046150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.065058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/163] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:23:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.068330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.159946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 76/164] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:00.160952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.204340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.250877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e638bbf-6a87-43de-bc8f-52d8cf0e4373 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.290533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.291351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.291634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.291908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.292783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:00.293088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.294754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.305249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/165] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.306153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.356059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11baf6-f699-43fa-b27d-40ecbc3c14bf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.388597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.429070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.429969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.430244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.430514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.431714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.433227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.436724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 77/166] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.439582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.482108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0363817d-ccdf-45a5-aec6-c61ac0bdd932 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.551762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.552616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.552895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.553168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.555868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.557354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.563117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/167] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.565375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.623202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.640116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e79418-a87f-494a-82ce-a444afc0ea33 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.681642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.682452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.682729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.683000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.694038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:00.701927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:00.702802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:00.703589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 78/168] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:00.704746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.732517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57185adb-d0cd-4124-91bc-b61406f4f7ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.768409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.798006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.798800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.799078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.799345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] PATH_INFO: `/v3/users/dac17cc0cb2947d29561d6c51e66f189` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.800564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] RBAC: Authorizing `identity:get_user(user_id=dac17cc0cb2947d29561d6c51e66f189)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.802215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.803794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/169] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/users/dac17cc0cb2947d29561d6c51e66f189 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.809169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.847808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39577627-cdb8-4f03-944c-5fe62eb57bab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:00.904938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:00.907199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:00.907701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:00.910333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:00.911858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:00.914513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:00.916277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:00.927399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 79/170] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:00.933727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:00.977477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:00.986101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4b57-d8b8-432b-8016-f5e7785efa51 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.028586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.029285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.029563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.029837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.118364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.137264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b2004-a4a6-418f-86bf-228b648b946a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.146887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.148763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e5b4b-314a-4ac2-8ff9-d58560ef681a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.199030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.199718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.199995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.200284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.201051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:01.201352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.202773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.220196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 80/171] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:01.221077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.318526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/172] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:01.319914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4681e-7c30-4890-8b13-e6f7674e1084 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.320225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4681e-7c30-4890-8b13-e6f7674e1084 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.320502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4681e-7c30-4890-8b13-e6f7674e1084 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.321910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/173] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:01.324815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4681e-7c30-4890-8b13-e6f7674e1084 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.329611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.362264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a5f2c-465f-4909-bff9-87addc3de44d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.405838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.407689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.407983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.408292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.409736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.411888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.413842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 81/174] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:01.415414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.443566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.460621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4681e-7c30-4890-8b13-e6f7674e1084 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.515921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.517080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.517507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.517937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] PATH_INFO: `/v3/users/d4c637fab90949129383c9f979fffa9f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.521173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] RBAC: Authorizing `identity:get_user(user_id=d4c637fab90949129383c9f979fffa9f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.522841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e0d80-6ef4-48bd-9d48-21f784d07420 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.524729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/175] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/users/d4c637fab90949129383c9f979fffa9f => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:01.526128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.526575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.527002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.554804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:01.555748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:01.557341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30f71f-79b9-4f9b-b8cc-037c2d8ee245 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.610142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.611260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.611708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.612164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.613818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.615918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.618001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 82/176] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:01.618975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.639962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.645451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/177] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:01.647197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.741875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128786e-9204-4ed6-bb98-7aebb55f9ab2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.757755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.774436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b69b2-ace9-49de-a424-440305bd1dbd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.778552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.779546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.779891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.780250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/dac17cc0cb2947d29561d6c51e66f189/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.782658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user_id=dac17cc0cb2947d29561d6c51e66f189, role_id=d7a3c0cbeb0b421aa95a7ec7ef07991c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.784499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.799065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 83/178] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:01 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/users/dac17cc0cb2947d29561d6c51e66f189/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:01.801191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.811540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.812260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.812546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.812808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.889159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.890681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb6065-fd5f-4238-8ef0-42be7a057c0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.891576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:01.929039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e828373-5cd9-4ebb-9819-77be73b1a097 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:01.965740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:01.966415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:01.966700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:01.966973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:01.967827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:01.968161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:01.969757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:01.984795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 84/179] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:01.985986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:01.990419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.008071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/180] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:02.012212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0c0a7-9e9a-4791-bf53-489cb37feb4b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.012212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0c0a7-9e9a-4791-bf53-489cb37feb4b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.012212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab0c0a7-9e9a-4791-bf53-489cb37feb4b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.012212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/181] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:02.012658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db55f-a900-4fc5-83ee-31bbaeb82171 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.013190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db55f-a900-4fc5-83ee-31bbaeb82171 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.013190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2db55f-a900-4fc5-83ee-31bbaeb82171 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.014722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/182] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:02.016109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2db55f-a900-4fc5-83ee-31bbaeb82171 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.116062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cb21d-a08e-4fa1-bd76-e48349175211 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.154395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.155549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.155997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.156465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.160020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.162217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2db55f-a900-4fc5-83ee-31bbaeb82171 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.173242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1c685ab2-b55f-4d74-ab16-c465a44ff0ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:02.184554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:02.185489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:02.186275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 85/183] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:02.187642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.187642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.191897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.211221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.212222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.212472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.212589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:02.212749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] PATH_INFO: `/v3/projects/72bae80a77264968a3d4dc1212893568` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.213784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:02.213983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] RBAC: Authorizing `identity:get_project(project_id=72bae80a77264968a3d4dc1212893568)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.215455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a415f5-0d9c-41b0-9752-8a2b4910f391 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.217547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/184] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/projects/72bae80a77264968a3d4dc1212893568 => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:02.218909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.219191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.219462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.240557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:02.242186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:02.290266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 86/185] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:02.291220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.309860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.334928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/186] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:02.336523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.398140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a44d13-3e72-40f2-a24a-b4151344f7ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.425018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.436405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.437092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.437373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.437646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.438402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:02.438709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.440124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.453068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 87/187] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:02.453068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.458417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be15a1-0ee9-46cb-a474-0a6b71613cb6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.497353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.498273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.498726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.499189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.500226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=72bae80a77264968a3d4dc1212893568, user.id=d4c637fab90949129383c9f979fffa9f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:02.500542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.502010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.516055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/188] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/role_assignments?user.id=d4c637fab90949129383c9f979fffa9f&scope.project.id=72bae80a77264968a3d4dc1212893568&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:02.516055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.573909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f72e826-fe14-4911-b377-735d1ed32fa9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.576332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.616897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.617882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.618228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.618572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.631133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:02.639749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:02.640851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:02.641652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 88/189] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:02.642314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.645396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b983e383-3872-43df-83f2-a07d8d5a290c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.692413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.693512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.693894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.694267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.695602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.698297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.704365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/190] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:02.705535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.708362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.783973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a23c80-0e64-479e-86b3-d786b1ba8084 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.840381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.840884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.841577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.841991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.847787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44592d3d-3ecd-4801-86d0-1f5dd0bcc848 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.858453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.859932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.861737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 89/191] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:02.862911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.870395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:02.891394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:02.892094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:02.892373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:02.892641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:02.893492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:02.893902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:02.895373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:02.904668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/192] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:02.904668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:02.965212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ec150-2e17-464b-809e-7f9dd7939c0e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:02.972490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.001700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.002405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.002659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.002947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.003648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.005398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.011702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2714a-5a80-4684-9b8a-82601b1bd44d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.026735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 90/193] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:02 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:03.029587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.065130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.065882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.066292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.066773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.141073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.174056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.175964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.199769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca97446-b0af-4e36-8f65-2b72f1b8ee97 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.241736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.242872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.243292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.243707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.245087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.246718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.248745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 91/194] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:03.253917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.287879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.298443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/195] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:03.304718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.378093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0b39b-908b-4162-8caa-95ae7df40cab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.405947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.418998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.419897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.420798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.421534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.439257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05385a9a-420f-4d15-8aaa-13ad8915fe17 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.483194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.484194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.484391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.484644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.486055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.487299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.490059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/196] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:03.495608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.510876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.512942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.551247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.666539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fc9b2-d6e6-431d-bcfa-b68a17f6b0da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.674765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 92/197] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:24:03.680379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.713475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.714593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.715068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.715557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.716702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=da66945243dd4cb1ab0cd4ad4358e56e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:03.717208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.718864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.726312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/198] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/role_assignments?user.id=da66945243dd4cb1ab0cd4ad4358e56e&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:03.801715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c28b6d-3045-4e72-b991-179d183bfb63 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:03.843978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.851511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:03.852431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:03.852703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:03.852967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:03.856877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:03.858387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13dd81-d95b-44d4-9f08-e0e474a125bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:03.860430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 93/199] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:03.866586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:03.886887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:03.991228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0fa6a-cbe3-40ef-b82c-7c0c75acdf1f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:04.025950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:04.027399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:04.027927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:04.028437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:04.029461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=da66945243dd4cb1ab0cd4ad4358e56e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:04.029965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:04.031636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c0073-057e-4560-815b-2f36254c02d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:04.038464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/200] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:03 2022] GET /identity/v3/role_assignments?user.id=da66945243dd4cb1ab0cd4ad4358e56e&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:04.426671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bdbbe-87c2-4c1d-a6d1-71bef54f1fad None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:04.427155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bdbbe-87c2-4c1d-a6d1-71bef54f1fad None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:04.427155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1bdbbe-87c2-4c1d-a6d1-71bef54f1fad None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:04.428777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 94/201] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:04.434354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:04.434636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:04.434910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:04.451704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:04.452897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:04.549824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/202] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:04.572578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bdbbe-87c2-4c1d-a6d1-71bef54f1fad None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:04.718547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1bdbbe-87c2-4c1d-a6d1-71bef54f1fad None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:04.752890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:04.754057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:04.754527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:04.754960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:04.765330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:04.773294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:04.774190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:04.774983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:04.777492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 95/203] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:04.783330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:04.796056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:04.885734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486c344-2f98-4936-b9b7-2b97f46aba2a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:04.929865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:04.930779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:04.931221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:04.931629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:04.932546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:04.932982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:04.934499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:04.943461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/204] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:04.949192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:05.043056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:05.087109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172ae43-962d-45cf-a833-9928d91f335e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:05.122659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:05.123544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:05.123818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:05.124115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] PATH_INFO: `/v3/users/dac17cc0cb2947d29561d6c51e66f189` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:05.125310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] RBAC: Authorizing `identity:get_user(user_id=dac17cc0cb2947d29561d6c51e66f189)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:05.126772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:05.129052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 96/205] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:04 2022] GET /identity/v3/users/dac17cc0cb2947d29561d6c51e66f189 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:05.134631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:05.256789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:05.303011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905979d-0a9e-40bb-9c58-ddfeba84106d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:05.354061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:05.354716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:05.354991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:05.355262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:05.449080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:05.450584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:05.535789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:05.582116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/206] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:05.587672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:05.741685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683aa7ad-9ad3-4579-bb78-006141968ddd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:05.778087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:05.797806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:05.798878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:05.799155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:05.799425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:05.804772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:05.806238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077f840-1b5f-42b7-9937-a7e3bc63aa52 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:05.808305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 97/207] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:05 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:05.814152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:05.937209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7dbd4-132f-4cb3-a9d9-a2f307bc8b2e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:05.974088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:05.974754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:05.975031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:05.975303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:05.976165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, user.id=dac17cc0cb2947d29561d6c51e66f189) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:05.976475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:05.977956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c34271-517b-445c-a81c-beecbbe26e84 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:05.984409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/208] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:05 2022] GET /identity/v3/role_assignments?user.id=dac17cc0cb2947d29561d6c51e66f189&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:06.038877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa46-a25b-40f3-b8f8-2623f349ce26 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.039162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa46-a25b-40f3-b8f8-2623f349ce26 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.039431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1afa46-a25b-40f3-b8f8-2623f349ce26 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.041074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 98/209] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:06.048575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89195d-aac7-4206-bfe3-689f999ee270 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.048751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89195d-aac7-4206-bfe3-689f999ee270 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.049039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89195d-aac7-4206-bfe3-689f999ee270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.081394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4a89195d-aac7-4206-bfe3-689f999ee270 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:06.082714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a89195d-aac7-4206-bfe3-689f999ee270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:06.110427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.160634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/210] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:06.170398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b36f8d-59c0-4bed-a3ae-e3e9b4d3d358 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.170701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b36f8d-59c0-4bed-a3ae-e3e9b4d3d358 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.170947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b36f8d-59c0-4bed-a3ae-e3e9b4d3d358 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.172955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 99/211] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:06.178613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237b30-128e-476b-9a0f-92d1a51cc8df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.178894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237b30-128e-476b-9a0f-92d1a51cc8df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.179170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29237b30-128e-476b-9a0f-92d1a51cc8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.197173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29237b30-128e-476b-9a0f-92d1a51cc8df None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:06.198250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b36f8d-59c0-4bed-a3ae-e3e9b4d3d358 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.200217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29237b30-128e-476b-9a0f-92d1a51cc8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:06.278660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/212] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:06.279994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feb43-5699-4bdb-8c93-1817f6923f0a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.280320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feb43-5699-4bdb-8c93-1817f6923f0a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.280591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1feb43-5699-4bdb-8c93-1817f6923f0a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.284460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/213] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:06.287864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.288543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.288814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.309768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:06.310868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:06.349621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b36f8d-59c0-4bed-a3ae-e3e9b4d3d358 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.390330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/214] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:06.396128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.396128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.396128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.396128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.396887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.398005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:06.399467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:06.401896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 100/215] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:06.407218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.423888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.471589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.512318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8447885-1165-4c8e-ab56-d8ae6a3d1c1f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.512822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12906e-d019-41b8-935d-5604eea9d15a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.546004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.547001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.547268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.547535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.548117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.548252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:06.548924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.549203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.549477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.549709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:06.560856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:06.568419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:06.569359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:06.570168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/216] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:06.571586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.571716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 101/217] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:06 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:24:06.577507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.683767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.710870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d52835-f979-4875-8afc-d420176472e2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.717495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a179-b4e6-4c49-8585-04b17b7fb333 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.742132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.754143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.756384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.756902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.757342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.765314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.766012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.766295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.766563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.767554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:06.767858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:06.769797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:06.771462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:06.779825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 102/218] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:06.780290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:06.781218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:06.782000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:06.783033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/219] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:06.785838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.789315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.905587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad1638-73cd-494e-9183-f472389662b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.927069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d81c3f-33d7-45a2-b098-e8b9c97d347a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:06.944804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.945698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.945976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.946248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] PATH_INFO: `/v3/users/dd991b20472547cba7ba7d6c6c6b336a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.964166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] RBAC: Authorizing `identity:get_user(user_id=dd991b20472547cba7ba7d6c6c6b336a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:06.966295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a60f26-1537-496c-b6dc-51351a082c20 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:06.968183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 103/220] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/users/dd991b20472547cba7ba7d6c6c6b336a => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:06.969521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a3b-2707-4224-94c8-af08d16bd1c5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.969755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:06.969896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a3b-2707-4224-94c8-af08d16bd1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.970108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e0a3b-2707-4224-94c8-af08d16bd1c5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.970437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.970708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.970979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.971611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 104/221] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:06.971719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:06.972012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:06.973589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:06.978330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:06.978617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:06.978891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:06.987158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/222] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:06.988200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:06.990963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:06.997107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:06.997503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:07.042013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.095143 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 105/223] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:07.096428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.105554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aa91a-508f-471d-9131-3a04361e1cbf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.142327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.143074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.143357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.143632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.218453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686707-0246-4c9a-bd8d-28a660b3e50c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.229009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.231125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.233513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.263524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.265051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.265333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.265840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.267282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.269542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.272124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 106/224] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:07.272937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.328750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.354499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/225] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:07.355422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.417234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c4fd8-a2f2-469a-8cfe-b4240267051a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.463617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.464493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.464772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.465043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.471340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e13de-c43f-46bb-9f1b-7ea404c1eb73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.473643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.476751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:07.485099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:07.486013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:07.486817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 107/226] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:07.487465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.522742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.524655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.525035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.525404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.526755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.528373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.535496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/227] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:07.535496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.604325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.612134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d287e4-b9c9-45a8-bb60-d7342bb4d369 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.649325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.650406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.650790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.651176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.652526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.654108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.656331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 108/228] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:07.657269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.657599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00f399-60f5-4bff-834d-d1e07d17b5cf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.710051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.711195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.711692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.712259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.713303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:07.713864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.715662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.725116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/229] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:07.726370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.760736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.780176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346121d-e3ef-42a8-935c-0d8f887b32ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.822566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.823462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.823733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.824000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/da66945243dd4cb1ab0cd4ad4358e56e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.826471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, user_id=da66945243dd4cb1ab0cd4ad4358e56e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.828282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.841000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a41cf7-fee8-42a7-8fd4-9b21c2f0e6c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:07.846703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 109/230] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:07 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/da66945243dd4cb1ab0cd4ad4358e56e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:07.847804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:07.905052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:07.921815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:07.924210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:07.926607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:07.927284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:07.929517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=dd991b20472547cba7ba7d6c6c6b336a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:07.931379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:07.935783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:07.950247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/231] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/role_assignments?user.id=dd991b20472547cba7ba7d6c6c6b336a&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:08.040085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8f9df-bfe9-4a8c-9cb5-1f1f67aceb05 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:08.082880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:08.084319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:08.084834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:08.085308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:08.086876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:08.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:08.090574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 110/232] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:07 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:08.096365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:08.222020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:08.229869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be69d7f-3c87-4877-85dd-4f4cbff6665b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:08.275409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:08.276791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:08.277274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:08.277742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:08.281425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:08.283190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487ecbc-3bc0-458f-a8f7-16898179eb8f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:08.285231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/233] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:08 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:08.291257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:08.394888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:08.403536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c109e44-3891-4a3f-84b5-8aaf1b9abfcb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:08.448155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:08.449147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:08.449430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:08.449701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/da66945243dd4cb1ab0cd4ad4358e56e/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:08.454228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, user_id=da66945243dd4cb1ab0cd4ad4358e56e, role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:08.455698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82b6fc-521e-4b9f-b158-991f1b5c2c77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:08.481094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 111/234] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:08 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/da66945243dd4cb1ab0cd4ad4358e56e/roles/85a7aff706174bb09e06d1ca1d363973 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:09.204178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68ec77-4c07-40ee-97cb-075b3016814c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.204523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68ec77-4c07-40ee-97cb-075b3016814c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.204740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68ec77-4c07-40ee-97cb-075b3016814c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.206382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/235] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:09.212326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929058-9cc4-4694-8cdf-d05f03847c91 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.212789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929058-9cc4-4694-8cdf-d05f03847c91 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.212978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0929058-9cc4-4694-8cdf-d05f03847c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.231061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c0929058-9cc4-4694-8cdf-d05f03847c91 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:09.232299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c0929058-9cc4-4694-8cdf-d05f03847c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:09.325409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 112/236] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:09.348725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68ec77-4c07-40ee-97cb-075b3016814c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:09.442854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb55-4d53-4803-9722-72db577fea7e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.443185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb55-4d53-4803-9722-72db577fea7e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.443401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73eb55-4d53-4803-9722-72db577fea7e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.445101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 113/237] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:09.452427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.452886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.453281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.483039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68ec77-4c07-40ee-97cb-075b3016814c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:09.485661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:09.487805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:09.545641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:09.547103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.547555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.547966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.584940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:09.593267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:09.594201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:09.595029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:09.595670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 114/238] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:09.602296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/239] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:09.607746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:09.623317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:09.626580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:09.696302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:09.813824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ebf01-abd6-438b-9b06-4e5ebcbeba7b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:09.825868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f86573-c8df-46b9-aa09-2de912e1590c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:09.864082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:09.865133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.865633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.866100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.867057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:09.867554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:09.869168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:09.878284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 115/240] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:09.887373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:09.898658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:09.899545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:09.899820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:09.900138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:09.901363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:09.902846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:09.904509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/241] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:09.910585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.055716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.059121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.077459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34d5f6-b0d1-459f-84bd-3a93f5e87c5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.105113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99462ff-84a9-4518-8234-e1fdfd71fd14 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.142056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.143345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.143850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.144583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] PATH_INFO: `/v3/users/a1a6a6a74ec541fcbed02f0f691656f4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.158799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a6a6a74ec541fcbed02f0f691656f4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:10.160706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:10.163034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 116/242] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:09 2022] GET /identity/v3/users/a1a6a6a74ec541fcbed02f0f691656f4 => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:10.170817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.174377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.175072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.175354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.175820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.176606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:10.178149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4cc1-b1d7-48ad-982f-dbb253f499f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:10.208986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/243] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:09 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:24:10.314732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c442310-d786-4264-a0b2-6ba06c135317 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.366879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.371928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.372629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.372965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.373282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.397167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b70a0-aae6-45c1-b93c-3d8f4d63ff51 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.397698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b70a0-aae6-45c1-b93c-3d8f4d63ff51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.397698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b70a0-aae6-45c1-b93c-3d8f4d63ff51 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.399452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/244] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:10.405206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.405488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.405758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.429652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:10.430894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:10.500745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:10.502243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:10.550607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/245] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:10.573781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.630135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 117/246] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:10.635864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.678110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.698358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.712107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ce170-fa80-4c59-9dd9-987cee62042c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.750031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.750846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.751127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.751400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.766346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3635342c-fb27-4a41-874c-555c60a006ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:10.775992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:10.777177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:10.778191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:10.780732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/247] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:10.782214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135e27-4057-4b3e-815b-541e013d8f49 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.782741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135e27-4057-4b3e-815b-541e013d8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.783157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135e27-4057-4b3e-815b-541e013d8f49 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.784901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/248] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:10.786932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51135e27-4057-4b3e-815b-541e013d8f49 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.796534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684b13f-f974-4acb-80b9-f99c5939acd3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.834025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.834907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.835184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.835459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.836704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:10.838152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d738b90-6775-4e2d-a868-f5eb220779b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:10.839884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 118/249] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:10.841223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.841506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.841778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.863561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:10.864872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:10.915375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51135e27-4057-4b3e-815b-541e013d8f49 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:10.950692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 119/250] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:10.952058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:10.964761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:10.965504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:10.965784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:10.966054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:10.966815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:10.967129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:10.968830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:10.979011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.986465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:10.988487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/251] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:10.995666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.069091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc0595b-1d06-4b50-b289-89b7b33efc2e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.110238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105ddf0-ba91-46d7-a82d-7db42ed53a92 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.114855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.115528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.115801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.116090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.116937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=a1a6a6a74ec541fcbed02f0f691656f4) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:11.117244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.118810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.130929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 120/252] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/role_assignments?user.id=a1a6a6a74ec541fcbed02f0f691656f4&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:11.133163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.154887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.155836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.156448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.156730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.182409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:11.197268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:11.200780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:11.203965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:11.205609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/253] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:11.211316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.303395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:11.311210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fca756-0058-433a-b1e4-8b629e38beb1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.321313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:11.360240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.363372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.363372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.363372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.364166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.365701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 121/254] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:11.373880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.410451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce01e-2461-4e5f-ba27-a8cefdf5b956 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.493366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.493958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.494240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.495281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.495479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:11.500067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.500067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.519211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c211c-b5af-4693-86d5-6d8e4d7871a4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.523490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/255] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:11.531516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.572548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.573583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.574013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.574432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.628148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:11.677960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.680898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db118c7-cf30-4d03-be7e-9839c6ecc840 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.688240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfae10c-ff8f-4421-86c1-744482c4e6c4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.715057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:11.729031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.730316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.730788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.731270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] PATH_INFO: `/v3/users/dd991b20472547cba7ba7d6c6c6b336a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.732824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] RBAC: Authorizing `identity:get_user(user_id=dd991b20472547cba7ba7d6c6c6b336a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.734620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd37a7-63f6-4247-bedd-791372a2e760 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.736641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/256] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/users/dd991b20472547cba7ba7d6c6c6b336a => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:11.741914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0062c6-fb67-4dd1-b852-91e654dac92c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.742460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0062c6-fb67-4dd1-b852-91e654dac92c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.742925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0062c6-fb67-4dd1-b852-91e654dac92c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.746274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/257] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:11.751114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0062c6-fb67-4dd1-b852-91e654dac92c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:11.809624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 122/258] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:11.809624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.809624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.809624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.848272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:11.850211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:11.893675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0062c6-fb67-4dd1-b852-91e654dac92c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:11.937243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:11.938509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:11.938988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:11.939455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:11.940902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:11.943427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:11.945561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/259] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:11.951297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.006230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:12.026538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:12.028178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 123/260] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:12.029149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.112052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e54c6a-dc47-45bf-8f63-246576c420d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.141710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95c8cb-bae8-4324-836d-586d8c303b77 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.166432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.167812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.168329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.168827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.170255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:12.172417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:12.174428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/261] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:11 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:12.175590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.198116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.199177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.199720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.200234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/dd991b20472547cba7ba7d6c6c6b336a/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.202684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, user_id=dd991b20472547cba7ba7d6c6c6b336a, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:12.204431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:12.223731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 124/262] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:12 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/dd991b20472547cba7ba7d6c6c6b336a/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:12.225069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.338493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:12.349574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a117d-4790-4c6e-ac2b-48b8da76443d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.401486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:12.419420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.420292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.420572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.420843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.435973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a422a-52ab-4e86-92aa-2ea781ba25ab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.447254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:12.455370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:12.456361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:12.457258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:12.460243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/263] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:12.465051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.503118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.504419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.504972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.505437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.506561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=da66945243dd4cb1ab0cd4ad4358e56e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:12.507076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:12.508802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:12.522720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 125/264] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:12 2022] GET /identity/v3/role_assignments?user.id=da66945243dd4cb1ab0cd4ad4358e56e&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:12.644279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec467-bc8b-41e1-94ac-3cf84137c4bc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.699725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.700451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.700729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.700999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.701754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:12.702064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:12.703485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:12.717910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/265] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:12.721510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.844617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f92f0-be46-4505-a9f0-fc7ad1894bec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:12.869574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:12.897109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:12.897954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:12.898235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:12.898509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] PATH_INFO: `/v3/users/da66945243dd4cb1ab0cd4ad4358e56e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:12.899753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=da66945243dd4cb1ab0cd4ad4358e56e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:12.901257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:12.903222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 126/266] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:12 2022] GET /identity/v3/users/da66945243dd4cb1ab0cd4ad4358e56e => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:12.908677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:12.996466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:13.058183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a5fe7-611d-4d26-9110-f5fcff1988a2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:13.096070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:13.096728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.097004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.097326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.171428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:13.172893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa0293-fe50-4909-811b-38c57fe14e53 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:13.243753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:13.306759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/267] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:13.312487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:13.434100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efecd2-10ab-4129-a2fc-9530790fc418 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.434730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efecd2-10ab-4129-a2fc-9530790fc418 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.434730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efecd2-10ab-4129-a2fc-9530790fc418 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.436287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/268] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:13.441379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.441909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.442190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.447005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baac42-5461-4283-9912-52b4e14c7ba2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:13.477060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:13.478269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:13.484528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:13.485522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.485801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.486073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.487286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:13.489511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:13.491707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 127/269] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:13 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:13.497006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:13.556052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:13.602689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/270] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:13.611579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208856f9-7189-40f4-bf3e-a9121e93350d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:13.646422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:13.649178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:13.650637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.650637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.650637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.651297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=da66945243dd4cb1ab0cd4ad4358e56e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:13.651610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:13.653125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:13.659910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 128/271] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:13 2022] GET /identity/v3/role_assignments?user.id=da66945243dd4cb1ab0cd4ad4358e56e&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:13.755506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:13.788711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93da135-f6d8-4a61-b395-f7185e6de199 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:13.844306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:13.845132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:13.845412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:13.845684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:13.862255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-258f5bbb-cc42-4606-98f8-ff7ea6250f23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:13.871152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:13.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:13.872929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/272] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:13.880742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.029177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414529fd-b678-4fd0-aa35-ce1c63167aa4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.048152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:14.072165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.072909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.073189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.073466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.074278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:14.074588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:14.075983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:14.086029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 129/273] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:14.088617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988058-75ab-4ca3-8ca7-eff40bc5f117 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.089133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988058-75ab-4ca3-8ca7-eff40bc5f117 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.089598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74988058-75ab-4ca3-8ca7-eff40bc5f117 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.091302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.092371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/274] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:14.097756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.098288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.098758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.118020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:14.120523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:14.191623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/275] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:14.205721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c562d-4d14-4367-b6b1-260e83a533f0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.216053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.244538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.245523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.245893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.246170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] PATH_INFO: `/v3/users/b3df23df657e42daa988c70c89a423f2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.263264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] RBAC: Authorizing `identity:get_user(user_id=b3df23df657e42daa988c70c89a423f2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:14.265024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:14.267155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 130/276] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:14 2022] GET /identity/v3/users/b3df23df657e42daa988c70c89a423f2 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:14.273489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.285599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:14.313119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:14.321383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1e054-3e2a-4038-ae15-256d95fd525c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.357450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.358258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.358535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.358803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.359946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:14.361441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:14.363310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/277] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:14.368594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.389276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb411c44-24f4-4c7f-9b43-2434a26719e3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.427153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.427821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.428140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.428416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.429306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, user.id=b3df23df657e42daa988c70c89a423f2, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:14.429617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:14.431131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79015074-2848-4f8e-bdf1-8e770e99f003 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:14.436661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 131/278] 10.222.0.55 () {56 vars in 1308 bytes} [Tue May 24 15:24:14 2022] GET /identity/v3/role_assignments?user.id=b3df23df657e42daa988c70c89a423f2&scope.system=all&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:14.496966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603092d0-ac46-4e02-a2f8-0d03635d512a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.539425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.540058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.540355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.540631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.541363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:14.542902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd2748-d783-48f6-9a09-c4d4d7f6392f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:14.552598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:14.576487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/279] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:14 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:24:14.687443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d709-0120-4428-a297-bc95d9bf59b2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.688733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d709-0120-4428-a297-bc95d9bf59b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.689207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3289d709-0120-4428-a297-bc95d9bf59b2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.691554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 132/280] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:14.697585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.698199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.698673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.717177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:14.718743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:14.795467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/281] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:14.820241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d709-0120-4428-a297-bc95d9bf59b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:14.932473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:14.938640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3289d709-0120-4428-a297-bc95d9bf59b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:14.973649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:14.974458 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:14.974731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:14.974996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:14.986550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-532b62a5-e96b-4956-9eeb-90d3e5fab937 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:14.994441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:14.995352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:14.996168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 133/282] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:15.001402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.019509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674ec94-3779-4183-851b-87b362f96676 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.019804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674ec94-3779-4183-851b-87b362f96676 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.020067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674ec94-3779-4183-851b-87b362f96676 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.021777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 134/283] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:15.033975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.034254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.034532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.053982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:15.055088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:15.115116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e05b23-0837-4200-850a-b19b588ea07a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.148246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:15.161485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 135/284] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:15.170187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.171174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.171454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.171721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.172492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:15.172788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:15.174202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:15.187831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/285] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:15.193121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.214849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.343272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:15.363051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2aac14-9bfd-4e8f-9d2b-531076c96e81 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.384493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a050dd-f8f9-44b1-ba61-58a46c6d2f47 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.417829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.418706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.418984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.419258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] PATH_INFO: `/v3/users/a1a6a6a74ec541fcbed02f0f691656f4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.420544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a6a6a74ec541fcbed02f0f691656f4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:15.422330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:15.424325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 136/286] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/users/a1a6a6a74ec541fcbed02f0f691656f4 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:15.433647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.452240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.453054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.453355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.453596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.481503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:15.493461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:15.494584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:15.500557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/287] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:15.513528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.560460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75126dd-99c5-4bc4-b472-e4ad0a354811 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.597011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.598307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.598720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.599122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.600845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:15.602574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:15.604685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 137/288] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:15.614082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.650297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:15.656469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:15.726964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07569233-7ba3-4a82-a232-826e14deffb4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.763843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.764771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.765049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.765321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/a1a6a6a74ec541fcbed02f0f691656f4/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.767471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, user_id=a1a6a6a74ec541fcbed02f0f691656f4, role_id=d7a3c0cbeb0b421aa95a7ec7ef07991c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:15.770011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:15.809118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326a283-9264-474f-9829-db71f6ad3515 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:15.810622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 138/289] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:15 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/users/a1a6a6a74ec541fcbed02f0f691656f4/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:15.848081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:15.848745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:15.849030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:15.849309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:15.850066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:15.850418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:15.851916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:15.861562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/290] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:15.867181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:15.976426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:15.992369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77dc3a3-12e9-4036-9471-8247c0f4a04e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:16.029631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:16.030714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:16.031091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:16.031461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] PATH_INFO: `/v3/users/dd991b20472547cba7ba7d6c6c6b336a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:16.032803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] RBAC: Authorizing `identity:get_user(user_id=dd991b20472547cba7ba7d6c6c6b336a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:16.034268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:16.036757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 139/291] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:15 2022] GET /identity/v3/users/dd991b20472547cba7ba7d6c6c6b336a => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:16.042434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:16.158042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8d34b-0632-4a21-9998-84c41100dba5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:16.165746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:16.196216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:16.196881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:16.197162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:16.197437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:16.268370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:16.270021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:16.369873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/292] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:16.375363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:16.477231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a87f1a-36a1-45ad-bc40-4a3c00d9e73f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:16.511779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:16.512689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:16.513247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:16.513526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:16.514761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:16.516284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554d923-9944-4b3d-b927-26c0a9c2a6bb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:16.518301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 140/293] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:16 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:16.524384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:16.537112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:16.664082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc287e9-4e9c-407c-b6b4-89942f61536f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:16.720754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:16.721418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:16.721695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:16.721960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:16.723082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=dd991b20472547cba7ba7d6c6c6b336a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:16.723386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:16.724870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821451-aac5-49a6-82c0-3af53554a00c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:16.731459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/294] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:16 2022] GET /identity/v3/role_assignments?user.id=dd991b20472547cba7ba7d6c6c6b336a&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:17.136882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1161-6085-4ff3-aa72-7d3426a8e3a1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.137388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1161-6085-4ff3-aa72-7d3426a8e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.137388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b1161-6085-4ff3-aa72-7d3426a8e3a1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.139038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 141/295] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:17.145736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e825-2328-4468-af89-296063543c5f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.146347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e825-2328-4468-af89-296063543c5f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.146818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65e825-2328-4468-af89-296063543c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.165106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4b65e825-2328-4468-af89-296063543c5f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:17.166609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b65e825-2328-4468-af89-296063543c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:17.233206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.242014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/296] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:17.262194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b1161-6085-4ff3-aa72-7d3426a8e3a1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.288185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.322271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fa6b6-8fa9-414f-a20e-f3490c937635 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.323580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fa6b6-8fa9-414f-a20e-f3490c937635 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.324117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fa6b6-8fa9-414f-a20e-f3490c937635 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.326363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/297] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:17.334245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.335422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.336224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.361104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:17.362093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:17.396968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3b1161-6085-4ff3-aa72-7d3426a8e3a1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:17.448768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:17.449575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.449849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.450112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.460230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/298] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:17.461928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:17.476058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:17.476948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:17.477733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 142/299] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:17.478376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.482605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.568472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.588534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f41-09c5-4bc0-be48-6f8d89c1ab42 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:17.591187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd970fd0-6f3c-4144-885e-5896cad50d81 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:17.620776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.624652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:17.625470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.625805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.626085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.627240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:17.627619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:17.628656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.628913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0cd21-40b0-4f11-a607-4ff72b777395 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:17.629239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.629647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.630546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 143/300] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:17.635123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:17.635123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:17.635293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229d98b-81b3-479c-913d-328553987f04 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.635293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229d98b-81b3-479c-913d-328553987f04 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.635293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0229d98b-81b3-479c-913d-328553987f04 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.635474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ba896-8147-4438-936f-fa1608473019 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:17.635540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 144/301] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:17.635876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229d98b-81b3-479c-913d-328553987f04 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.647233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/302] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:17.648441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.648725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.648995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.666273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:17.667444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:17.743143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/303] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:17.743988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.767022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0229d98b-81b3-479c-913d-328553987f04 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:17.801911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:17.802586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.802865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.803136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.803827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:17.805329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:17.824087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 145/304] 10.222.0.55 () {60 vars in 1134 bytes} [Tue May 24 15:24:17 2022] POST /identity/v3/groups => generated 229 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:17.824087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.846243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.860796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1045de-ade2-49bb-931c-c4978fa2988e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:17.887975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:17.910249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:17.911686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:17.912189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:17.913252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] PATH_INFO: `/v3/users/b3df23df657e42daa988c70c89a423f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:17.915361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] RBAC: Authorizing `identity:get_user(user_id=b3df23df657e42daa988c70c89a423f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:17.917457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:17.920658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/305] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:17 2022] GET /identity/v3/users/b3df23df657e42daa988c70c89a423f2 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:17.926644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:17.997832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfae28d-f90c-41fa-b301-e6a645604db0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.040958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.041500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.041780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.042045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.049506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935ae4a-3cbf-4f3b-be9c-4cb950e3102e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.058523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:18.069107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:18.070050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:18.070849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:18.075462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 146/306] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:18.076869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:18.096224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.097525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.098171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.098171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] PATH_INFO: `/v3/system/users/b3df23df657e42daa988c70c89a423f2/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.099618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3df23df657e42daa988c70c89a423f2, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:18.101085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:18.114429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/307] 10.222.0.55 () {58 vars in 1334 bytes} [Tue May 24 15:24:17 2022] PUT /identity/v3/system/users/b3df23df657e42daa988c70c89a423f2/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:24:18.163320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:18.199203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aa9b1-9848-4419-8190-a2e11abe9dff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.236815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.237571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.238017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.238827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.239231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:18.239564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:18.241007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:18.255463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 147/308] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:18.256918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:18.326904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:18.379453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04778307-e8b2-48a5-909b-4a1e0cdaa5d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.419829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.421296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.421716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.422125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] PATH_INFO: `/v3/users/c46a0d36d70345a28922715ce6f95175` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.436357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] RBAC: Authorizing `identity:get_user(user_id=c46a0d36d70345a28922715ce6f95175)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:18.438029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1e9d9-6ab8-42d9-a89c-25519113ad7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:18.440220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/309] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/users/c46a0d36d70345a28922715ce6f95175 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:18.448073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:18.502298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701084-bc7f-412b-ae2a-6499e0bb6a57 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.502578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701084-bc7f-412b-ae2a-6499e0bb6a57 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.502853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a701084-bc7f-412b-ae2a-6499e0bb6a57 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.504720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/310] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:18.509729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.510013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.510278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.529258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:18.530725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:18.555136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e5e87-f24c-4035-a775-438ad3c58cbb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.589997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:18.597556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.598352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.598678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.599042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.599972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, user.id=c46a0d36d70345a28922715ce6f95175, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:18.600307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:18.601755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:18.607451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/311] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:18.620211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 148/312] 10.222.0.55 () {56 vars in 1308 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/role_assignments?user.id=c46a0d36d70345a28922715ce6f95175&scope.system=all&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:18.635089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:18.685152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:18.746468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc7be05-2d10-4baa-aea2-7e069cfc804c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.782994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.783808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.784115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.784387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.797007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:18.805046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:18.805993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:18.806807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/313] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:18.812360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:18.915704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78359585-72d9-47a6-8081-939de9cc9471 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:18.953374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:18.954056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:18.954330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:18.954599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:18.955343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:18.955643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:18.957081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:18.965929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 149/314] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:18.971676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:19.075810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e735ddf-3f97-46c1-b0ae-63e5bbd10427 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:19.077434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:19.110477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:19.111367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:19.111639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:19.111910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] PATH_INFO: `/v3/users/a1a6a6a74ec541fcbed02f0f691656f4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:19.113144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a6a6a74ec541fcbed02f0f691656f4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:19.114649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:19.116386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/315] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:18 2022] GET /identity/v3/users/a1a6a6a74ec541fcbed02f0f691656f4 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:19.121971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:19.181891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:19.224404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f372d-bbdd-4f40-be9f-c7de36ff86b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:19.258291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:19.259017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:19.259293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:19.259566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:19.329184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:19.330741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:19.387914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:19.430797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 150/316] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:19.436478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:19.536378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df71b722-5335-48c5-a0b6-7b3ca64afe25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:19.570381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:19.571253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:19.571573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:19.571889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:19.573067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:19.574530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebc9a8-5552-4ce9-a40e-177924db86e9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:19.576390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/317] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:19 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:19.582440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:19.680978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e651ab9-11b1-436e-ab51-c21e72992439 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:19.714927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:19.715626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:19.715900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:19.716237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:19.717298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718, user.id=a1a6a6a74ec541fcbed02f0f691656f4) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:19.717603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:19.719064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ff7e2-b7fd-46d3-af8d-a916e8d911a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:19.725500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 151/318] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:19 2022] GET /identity/v3/role_assignments?user.id=a1a6a6a74ec541fcbed02f0f691656f4&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:20.215483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b5708-3045-411c-b493-c2a3ef4560d6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.216253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b5708-3045-411c-b493-c2a3ef4560d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.216253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b5708-3045-411c-b493-c2a3ef4560d6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.217608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/319] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:20.223381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed980-0681-4dde-abb8-e1f5396385ed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.223814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed980-0681-4dde-abb8-e1f5396385ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.223884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bed980-0681-4dde-abb8-e1f5396385ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.242631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f7bed980-0681-4dde-abb8-e1f5396385ed None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:20.243885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7bed980-0681-4dde-abb8-e1f5396385ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:20.265416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:20.321562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 152/320] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:20.343780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b5708-3045-411c-b493-c2a3ef4560d6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:20.389378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:20.432367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230cd4-f4a1-4494-80cc-a683aceaa5c6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.432653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230cd4-f4a1-4494-80cc-a683aceaa5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.432927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4230cd4-f4a1-4494-80cc-a683aceaa5c6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.434343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 153/321] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:20.439374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.439653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.439921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.452646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b5708-3045-411c-b493-c2a3ef4560d6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:20.456775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:20.457963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:20.489689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:20.490386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.490685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.490953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.492173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:20.493654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:20.495469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/322] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:20.500813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:20.530893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 154/323] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:20.554679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:20.604378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbac63-1710-4858-87da-daff895a2fa0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:20.611946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:20.640079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:20.640755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.641043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.641319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] PATH_INFO: `/v3/groups` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.641526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:20.642005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:20.643540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:20.662187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f86cc-55d1-4622-947a-219d1bb4359f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:20.665442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/324] 10.222.0.55 () {60 vars in 1134 bytes} [Tue May 24 15:24:20 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:24:20.715746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:20.716593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.716871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.717167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.729361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-342afbf6-a0e1-4736-b2ac-50a6e701ed49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:20.737507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:20.738648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:20.744311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 155/325] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:20.744985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:20.853379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b62dba4-bacb-476c-a2c6-58da25607dcc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:20.863217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85a70-f40b-4a3d-8156-643e357835ba None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.863217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85a70-f40b-4a3d-8156-643e357835ba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.863502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f85a70-f40b-4a3d-8156-643e357835ba None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.864985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 156/326] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:20.870312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.870592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.870881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.888821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:20.890008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:20.892422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:20.893184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:20.893911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:20.893911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:20.894602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:20.894904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:20.896317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:20.905851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/327] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:20.912885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:20.943710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:20.976274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 157/328] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:21.004129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:21.030707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08c85a-5b98-4919-a396-deb2836ba850 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.069995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.071043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.071319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.071620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] PATH_INFO: `/v3/users/b3df23df657e42daa988c70c89a423f2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.072847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] RBAC: Authorizing `identity:get_user(user_id=b3df23df657e42daa988c70c89a423f2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:21.074303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:21.076263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/329] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:20 2022] GET /identity/v3/users/b3df23df657e42daa988c70c89a423f2 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:21.081700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:21.082526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:21.123791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335bf4a-292d-4b7c-a939-1e5206ea96ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.171757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.172582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.172853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.173116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.187549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:21.196011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:21.196977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:21.197758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 158/330] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:21.203165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:21.226483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:21.236360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e2c72-2c7c-4ee5-b5f7-5389564e6e9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.279005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.279892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.280334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.280764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.281730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] RBAC: Adding query filter params (role.id=491d70ccf49b4888bfe6d30f063be88e, user.id=b3df23df657e42daa988c70c89a423f2, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:21.282178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:21.283797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:21.297475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/331] 10.222.0.55 () {56 vars in 1308 bytes} [Tue May 24 15:24:21 2022] GET /identity/v3/role_assignments?user.id=b3df23df657e42daa988c70c89a423f2&scope.system=all&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:21.317355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caee8b2-ad44-4a98-abb5-4e0a3cf6f9dd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.352442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:21.361844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.362711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.364275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.364587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.365386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:21.365695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:21.367160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:21.385881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 159/332] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:21.391679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:21.528541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd2403-4670-4c3a-b210-e7fa7a8ceb37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.564768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.565582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.565981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.566341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] PATH_INFO: `/v3/users/c46a0d36d70345a28922715ce6f95175` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.567598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] RBAC: Authorizing `identity:get_user(user_id=c46a0d36d70345a28922715ce6f95175)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:21.569087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffebbd3-3440-4d6a-a6cd-5dc8ddc9db50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:21.571025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/333] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:21 2022] GET /identity/v3/users/c46a0d36d70345a28922715ce6f95175 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:21.576531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:21.726780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b40a8-fca9-4c31-9c6d-d7c0d0c288b9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:21.763311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:21.764616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:21.765090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:21.765580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] PATH_INFO: `/v3/system/users/c46a0d36d70345a28922715ce6f95175/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:21.767256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c46a0d36d70345a28922715ce6f95175, role_id=d7a3c0cbeb0b421aa95a7ec7ef07991c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:21.769174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16bd4b-a26f-441f-b804-31f7382d00ae None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:21.779761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 160/334] 10.222.0.55 () {58 vars in 1334 bytes} [Tue May 24 15:24:21 2022] PUT /identity/v3/system/users/c46a0d36d70345a28922715ce6f95175/roles/d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:24:23.425811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a471c-717d-417e-a878-edb70e46572b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:23.426447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a471c-717d-417e-a878-edb70e46572b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:23.426447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a471c-717d-417e-a878-edb70e46572b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:23.428010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/335] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:23.433722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f063979-d1db-43b9-9961-7f545c9772d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:23.434572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f063979-d1db-43b9-9961-7f545c9772d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:23.435044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f063979-d1db-43b9-9961-7f545c9772d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:23.457850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8f063979-d1db-43b9-9961-7f545c9772d7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:23.459414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8f063979-d1db-43b9-9961-7f545c9772d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:23.526286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:23.541841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 161/336] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:23.583015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a471c-717d-417e-a878-edb70e46572b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:23.623023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:23.714666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058749-5405-4436-abdc-ceb0de953436 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:23.715295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058749-5405-4436-abdc-ceb0de953436 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:23.715791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87058749-5405-4436-abdc-ceb0de953436 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:23.717491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 162/337] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:23.722794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:23.723332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:23.723799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:23.731650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a471c-717d-417e-a878-edb70e46572b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:23.751968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:23.753541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:23.777693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:23.778612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:23.778802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:23.779055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:23.801453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:23.809433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:23.810373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:23.811189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:23.815091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/338] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:23.816342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:23.866747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 163/339] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:23.913515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:23.936450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:23.951456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3e8f0-0068-4eb2-86b4-73f8af377d9f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.002798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.003524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.003804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.004106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.004859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:24.005159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:24.006585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:24.015734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/340] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:24.026763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:24.149271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f717f7-2a86-41bb-90e5-6a1fbb488d90 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.198012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bea26b-7c20-48ce-8aa2-45c51becd124 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.200819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.201559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.201779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.202049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.235074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.241256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.291554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.292598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.292879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.293147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.301045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:24.302858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb86aa9-79f4-4afe-aa13-3c5dfe0a261a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:24.312570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b50be29a-dae9-408d-8083-7ddd7f91e164 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:24.321405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:24.322356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:24.323183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 164/341] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:24.325432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e31d-d5d4-4ea6-810f-2bed0547d88f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.325432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e31d-d5d4-4ea6-810f-2bed0547d88f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.325432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1e31d-d5d4-4ea6-810f-2bed0547d88f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.326780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 165/342] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:24.331013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1e31d-d5d4-4ea6-810f-2bed0547d88f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:24.509203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/343] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:24:24.510497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.510779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.511248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.517635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1e31d-d5d4-4ea6-810f-2bed0547d88f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.531504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:24.532992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:24.562247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.562927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.563204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.563473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.564783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:24.565096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:24.566758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43013a0-6e15-46ca-8464-de0f31db0f30 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:24.579610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 166/344] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:24.581377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7326-7256-4fee-b42c-616f19dd6e54 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.581674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7326-7256-4fee-b42c-616f19dd6e54 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.581942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7326-7256-4fee-b42c-616f19dd6e54 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.585267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 167/345] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:24.586010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7326-7256-4fee-b42c-616f19dd6e54 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:24.589687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.633623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/346] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:24.636845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:24.674755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.758108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7326-7256-4fee-b42c-616f19dd6e54 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.767821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca53eb5-0341-4f53-8047-4f8b85dede7f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:24.796360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.797613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.798110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.798563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.799956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:24.801981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f65aa-fc98-477f-970f-cb737c026992 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:24.804156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 168/347] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:24.804298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:24.805022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.805426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.805699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.806478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:24.806989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:24.807449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:24.824914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:24.826498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:24.852235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.880317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:24.881819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:24.902852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 169/348] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:24.904236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:24.948223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:24.989590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/349] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:24.990797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.016249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007e3cb-50b6-4ee2-becc-eb6b6926b3c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.054329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.056973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.057518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.057802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.072917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.076500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:25.086099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:25.087289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:25.088126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:25.092528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 170/350] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:25.092528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.109369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270847bb-37b7-45e5-a180-dc79623b5189 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.146056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.147344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.147803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.155028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.156880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.158855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/351] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:24 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.160420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.198332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.202242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63622aec-db82-447f-a0dc-1928156cbc07 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.239729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.240273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.240552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.240825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.251690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:25.259819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:25.260743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 171/352] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:25.262228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.274845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d950bc2-0b58-46c7-9c04-c831f97cc45e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.312588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.313957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.314467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.314975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.316837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.318745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.321953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/353] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.323175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.333744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.378922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1537-11a7-40d9-a1ef-a9d16232c0f0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.415383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.416441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.416882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.417393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.418482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:25.418947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.420966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.430196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 172/354] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.431350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.438452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed12c1-f1d4-4527-a428-0616a4bf2ad6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.456621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.479673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.480740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.481298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.481767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.482850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:25.483395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.485119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.491687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/355] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.494003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.551314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac89e92-f675-4cc7-8f0f-9a86c0ad43c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.574585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.591715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.592665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.592947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.593235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.593967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:25.594499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.595750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.605395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 173/356] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.606113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.611276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46bf71-ce5e-44fe-98a7-6d1cbe723361 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.651579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.652902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.653374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.653882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.655312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.657104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae5efa-7012-4697-b421-a59b47ce93d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.659057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/357] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.660897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4850af-c147-4aef-92fb-09fffd747554 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.661420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4850af-c147-4aef-92fb-09fffd747554 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.661876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4850af-c147-4aef-92fb-09fffd747554 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.663400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/358] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:25.664513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4850af-c147-4aef-92fb-09fffd747554 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.713218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.736688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ead0-3d0e-4d5a-899e-4b8a838cd652 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.781745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.781745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.781745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.781745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.799648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4850af-c147-4aef-92fb-09fffd747554 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:25.845721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:25.848002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:25.848489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:25.848905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] PATH_INFO: `/v3/users/c46a0d36d70345a28922715ce6f95175` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:25.850273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=c46a0d36d70345a28922715ce6f95175)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.851725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.853544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/359] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/users/c46a0d36d70345a28922715ce6f95175 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:25.854224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:25.863171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:25.882255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:25.883729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329876e0-3633-499e-b61f-38e2b232832d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:25.989329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2eaca-4443-40f4-85c0-f32e16ad0ab6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.016502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 174/360] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:26.017917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.018201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.018467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.041662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:26.042850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:26.044693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.045837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.046925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.047211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.049072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:26.049072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.049849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b810a5b-cb0b-4a9d-ab52-364795c07107 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.058377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/361] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:25 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.059683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c7c0-c0ab-4174-9bac-022c4e27f541 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.059964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c7c0-c0ab-4174-9bac-022c4e27f541 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.060990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c7c0-c0ab-4174-9bac-022c4e27f541 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.062439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/362] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:26.063095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065c7c0-c0ab-4174-9bac-022c4e27f541 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.148679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 175/363] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:26.149478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.156385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.199190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065c7c0-c0ab-4174-9bac-022c4e27f541 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.248003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.248717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.249005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.249282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.250472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] RBAC: Adding query filter params (role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c, user.id=c46a0d36d70345a28922715ce6f95175, scope.system=all) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:26.250829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.254459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974adf6d-3bd0-4621-83f7-6ed8307c8487 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.272779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/364] 10.222.0.55 () {56 vars in 1308 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/role_assignments?user.id=c46a0d36d70345a28922715ce6f95175&scope.system=all&role.id=d7a3c0cbeb0b421aa95a7ec7ef07991c => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.274546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.274848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.275126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.296061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024aa14-d863-4cf1-803f-637d7819ac5d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.296990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:26.298216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:26.314965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.338536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.339425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.339704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.339974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.341234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.342688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.344635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 176/365] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.345261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.374444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/366] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:26.375385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.437399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.487031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed784ac3-3299-4b0f-9ecb-f3c7cf449e40 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.498246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637afad-d825-4dba-9f72-43cf8dbc4db5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.529383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.530267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.530545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.530819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.532197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.533845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.535583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/367] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.536904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.584051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.584883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.585161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.585434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.629000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:26.637264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:26.638148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:26.638944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:26.644383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 177/368] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:26.645301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.646314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abff5a3-395d-4b9c-9c84-8ca32713d491 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.688517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.692952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.694162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.694822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.695096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.708870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:26.718041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:26.718941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:26.719726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/369] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:26.721034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.752635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.825607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06590d-a157-4f84-aa72-40325450436a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.826757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1cf588-7881-4cfc-bc63-0fd2d0d4e6cf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:26.861821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.862623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.863029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.863380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.864066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:26.864375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.865753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.866991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:26.868621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:26.869099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:26.869579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:26.870615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:26.871184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:26.872910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:26.874162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/370] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.875549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.884441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 178/371] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:26.893667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:26.929301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:26.985949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b1c59-fc21-4988-ba9d-f0bb91ddd088 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.023855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.024566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.024842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.025113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.025980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:27.026171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.027626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.037899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/372] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:27.043959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.074348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411ef76-d4d3-4db2-8b24-faa551c46e54 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.081335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.161544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ca4c0-dd67-4b54-b618-231a69068dad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.165399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.166077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.166362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.166635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.176446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.200615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.201797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.202136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.202467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.275860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.275860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.317945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.319666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.376091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/373] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:27.382272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.396988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.496358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e168c2-f826-430c-80d9-d2e55a09a328 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.531358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 179/374] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:27.537434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.539437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.540397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.541205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.541587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.541872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] PATH_INFO: `/v3/projects/9a64b5dceb3e44fc817c5b6566c89325` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.543246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] RBAC: Authorizing `identity:get_project(project_id=9a64b5dceb3e44fc817c5b6566c89325)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.544788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.547257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/375] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/projects/9a64b5dceb3e44fc817c5b6566c89325 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:27.552719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.715032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af1527-c574-40ec-a88b-7ce0dc1dc25c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.719507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.778421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8d98d-4b9e-458a-b7ce-6a6c9a5850c1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:27.801068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.801731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.802064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.802329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.804867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.806316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.809142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 180/376] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:27.816129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.848219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:27.849099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:27.849374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:27.849930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] PATH_INFO: `/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:27.857714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] RBAC: Authorizing `identity:get_group(group_id=9a69d469d489499ba2ba45cb8ddcecbe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:27.859216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:27.861187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/377] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:27.868772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:27.964390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:27.994134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545a3de-1972-4d40-a20c-b8f07194dd09 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:28.038307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:28.039193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.039467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.039732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.041581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:28.043032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:28.045205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 181/378] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:28.051242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:28.108768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c48b22-e59b-4c15-a1ce-7b3ff9331e24 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:28.170760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d76a28-c2e2-46dc-b194-dc1cd5987f58 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:28.212509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:28.213172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.213453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.213723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.214558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:28.214866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:28.216478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ea2327-01df-48f3-a31c-a5d058603fa1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:28.216826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:28.217490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.217769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.218038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.218925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] RBAC: Adding query filter params (group.id=9a69d469d489499ba2ba45cb8ddcecbe, role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=9a64b5dceb3e44fc817c5b6566c89325) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:28.219232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:28.222634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 182/379] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:28 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:28.229032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d79c89-5245-44f6-9924-ac0d44a75131 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:28.247300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/380] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:27 2022] GET /identity/v3/role_assignments?group.id=9a69d469d489499ba2ba45cb8ddcecbe&scope.project.id=9a64b5dceb3e44fc817c5b6566c89325&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 260 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:28.494106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff017a9-bd58-4f6d-924c-35a843aaf073 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.494388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff017a9-bd58-4f6d-924c-35a843aaf073 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.494665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff017a9-bd58-4f6d-924c-35a843aaf073 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.496291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 183/381] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:28.502413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.502726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.503062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.515812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:28.524335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:28.525810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:28.614533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/382] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:28.652922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff017a9-bd58-4f6d-924c-35a843aaf073 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:28.656894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:28.786720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff017a9-bd58-4f6d-924c-35a843aaf073 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:28.835391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:28.836244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.838974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.839269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.862689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-333a5171-dc9b-47b9-b259-daeb3a683e90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:28.871202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:28.872148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:28.872953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:28.876132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 184/383] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:28.878656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:28.896935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6a72-9150-44b9-8edc-f49eaad731ab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.897531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6a72-9150-44b9-8edc-f49eaad731ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.897806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6a72-9150-44b9-8edc-f49eaad731ab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.899472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 185/384] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:28.913737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:28.914031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:28.914271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:28.942258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:28.943462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:29.070157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.078937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f8e41-2f30-45eb-9556-34d521576869 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.090399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 186/385] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:29.120969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.125610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.126295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.126577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.126852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.128574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:29.128917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.130315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.132970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.140576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/386] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:29.146594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.270413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0e75-a4fa-429f-ae3d-d4ff035c443f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.308714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.309596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.309871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.310142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.311350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.313041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.314990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/387] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:29.325919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.336352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc826259-16f4-40b6-96fc-d9554698ebc1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.383828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.384776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.385056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.385330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.402748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:29.415856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:29.416841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:29.417618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:29.419977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 187/388] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:29.425043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.439578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.510729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae7ebfa-7e9e-4eed-a17d-19270c859306 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.532134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.574045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.574936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.575213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.575487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.578077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.579482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.583432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/389] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:29.593960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.616728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1bbfa3-6d1e-4fcc-9937-208bdd0b763a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.673763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.674445 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.674733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.675072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.675875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:29.676693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.678228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ea3bf-94f4-4d89-8629-cab280c6caba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.687394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 188/390] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:29.691032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3acaf-5100-4adb-98ea-58bad8cbb635 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.691431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3acaf-5100-4adb-98ea-58bad8cbb635 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.691835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3acaf-5100-4adb-98ea-58bad8cbb635 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.693477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 189/391] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:29.694216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3acaf-5100-4adb-98ea-58bad8cbb635 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.750874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4824d-b568-410d-b777-cb844809150e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.789572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.790529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.790901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.791324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.793689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, group_id=f6b9ac76e6a44beca4a27ad35c92865d, role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.795200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c573-6f89-4a24-8753-886079a74ec1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.798291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.819795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/392] 10.222.0.55 () {58 vars in 1442 bytes} [Tue May 24 15:24:29 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/85a7aff706174bb09e06d1ca1d363973 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:24:29.825655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.825943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.826343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.834799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3acaf-5100-4adb-98ea-58bad8cbb635 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:29.852146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:29.853765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:29.856894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:29.881224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:29.882398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:29.882398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:29.882664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:29.883927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:29.885400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:29.887254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 190/393] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:29.892642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:29.969853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/394] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:30.023852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:30.047257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6f3f-9657-4a93-8e77-ea9fc5b47b2e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.092451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.093286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.093564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.093902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.095272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:30.097421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:30.099726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 191/395] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:29 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:30.109070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:30.175364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:30.207405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275c1da-9814-4993-b7d7-62958a3a6960 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.229117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473f21e8-3a58-4501-99a7-e4c8662206af None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.239940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:30.253468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.254285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.254567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.254842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.266073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:30.267319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.268246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.268624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.269124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.271585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c, group_id=f6b9ac76e6a44beca4a27ad35c92865d, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:30.273078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:30.276349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:30.277416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:30.278447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:30.280132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/396] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:30.286733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:30.294372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 192/397] 10.222.0.55 () {58 vars in 1442 bytes} [Tue May 24 15:24:30 2022] PUT /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:30.411663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb2b21-dff6-421d-8008-ea8c220a9ea2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.450002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.450692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.450970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.451244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.452007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:30.452341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:30.453836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:30.464436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/398] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:30.473944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:30.607225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacee28-f5a6-4327-b4f6-c9184f5f93b7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.622450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:30.648360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.649246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.649525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.649802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.651094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:30.652574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:30.658693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 193/399] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:30 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:30.660772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:30.732774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:30.791417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f87b3f-7b15-4cdc-b0f7-7fb97755d595 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:30.831501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:30.832469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:30.832745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:30.833014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:30.834221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:30.835697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82563e2f-a0f2-4266-88d7-7aeb196df1f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:30.837738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/400] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:30 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:30.843127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.022810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332dffd-c34b-4918-b297-d48f5a257080 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.067368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:31.068303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.068580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.068847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.070828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, group_id=f6b9ac76e6a44beca4a27ad35c92865d, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:31.072303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66632968-f230-48e1-85d9-fe96920fc70e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:31.105253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 194/401] 10.222.0.55 () {58 vars in 1442 bytes} [Tue May 24 15:24:30 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:31.217504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82fa8a-1fe3-47fe-85a3-69bc019c2a53 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.217504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82fa8a-1fe3-47fe-85a3-69bc019c2a53 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.217504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb82fa8a-1fe3-47fe-85a3-69bc019c2a53 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.218561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/402] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:31.223004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e04f3-0eeb-49a4-ac7f-3757716790e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.223309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e04f3-0eeb-49a4-ac7f-3757716790e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.223561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e04f3-0eeb-49a4-ac7f-3757716790e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.249817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a65e04f3-0eeb-49a4-ac7f-3757716790e0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:31.251032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a65e04f3-0eeb-49a4-ac7f-3757716790e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:31.300113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:31.354934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 195/403] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:31.377822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82fa8a-1fe3-47fe-85a3-69bc019c2a53 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.380568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0922a-315a-4db7-818f-b68e5f44170d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.380855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0922a-315a-4db7-818f-b68e5f44170d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.381132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0922a-315a-4db7-818f-b68e5f44170d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.382816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 196/404] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:31.388113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.389596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.389596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.407141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:31.408358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:31.507515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 197/405] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:31.538802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb82fa8a-1fe3-47fe-85a3-69bc019c2a53 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.561719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:31.583306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.594249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:31.595086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.595395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.595635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.620078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:31.632521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:31.633415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:31.634187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:31.637015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/406] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:31.643108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.706868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:31.718324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d93bf-4b3e-4dc4-a410-cfd9573f1dc4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.777871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:31.778678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.778952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.779220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.800108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:31.803473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6adddd-7d14-4c9e-a23b-db88dc74910b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:31.804782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:31.805648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:31.806460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 198/407] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:31.817070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.842827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:31.844067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.844545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:31.845035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:31.845847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:31.846212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:31.847915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:31.872387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/408] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:31.873361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:31.912005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:31.937421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb2fc5-577f-443d-a713-d368d17642ca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.980915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce5761-2b3c-4d41-ba26-d4643faecacb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:31.994080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:31.995090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:31.995579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.000125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.001095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:32.001688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.003320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.015364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.016351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.016695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.017032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.018271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.019892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.021906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/409] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:31 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:32.022430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 199/410] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:32.027378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:32.027583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:32.093830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:32.130613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac6ae3-4b72-4258-922e-f2c9df60e512 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:32.141363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:32.189125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.189997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.190271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.190613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.191763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.193246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.195158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/411] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:32 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:32.203243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:32.245153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bded9a9-b68d-465b-a1a2-ea7b941181c4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:32.310880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.311751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.312054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.312314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] PATH_INFO: `/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.314629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] RBAC: Authorizing `identity:get_group(group_id=9a69d469d489499ba2ba45cb8ddcecbe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.316375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.322247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 200/412] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:32 2022] GET /identity/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe => generated 229 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:32.323716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:32.406068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf4dd6-6655-48a8-866d-bd88cdaab2ec None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:32.434396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:32.448858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83119adc-4d41-453f-bdac-c0ed225e9f90 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:32.486147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.487051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.487375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.487648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.494589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] RBAC: Authorizing `identity:create_grant(project_id=5a1e8ee61a454b19822b93f53b6d8718, group_id=f6b9ac76e6a44beca4a27ad35c92865d, role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.494589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ce5f-587e-4d63-b63d-b0d4b05784ee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.499955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.501014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.501394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.501615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] PATH_INFO: `/v3/projects/9a64b5dceb3e44fc817c5b6566c89325` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.503069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] RBAC: Authorizing `identity:get_project(project_id=9a64b5dceb3e44fc817c5b6566c89325)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.504700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.507556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 201/413] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:32 2022] GET /identity/v3/projects/9a64b5dceb3e44fc817c5b6566c89325 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:32.514940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:32.522648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/414] 10.222.0.55 () {58 vars in 1442 bytes} [Tue May 24 15:24:32 2022] PUT /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718/groups/f6b9ac76e6a44beca4a27ad35c92865d/roles/85a7aff706174bb09e06d1ca1d363973 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:32.664931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07376-f986-47b6-9477-ff1f66835066 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:32.707920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:32.708873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.709152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.709422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] PATH_INFO: `/v3/projects/9a64b5dceb3e44fc817c5b6566c89325/groups/9a69d469d489499ba2ba45cb8ddcecbe/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.717783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a64b5dceb3e44fc817c5b6566c89325, group_id=9a69d469d489499ba2ba45cb8ddcecbe, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:32.719280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a859b0-d582-4108-818a-6e54423a1027 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:32.750759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 202/415] 10.222.0.55 () {58 vars in 1442 bytes} [Tue May 24 15:24:32 2022] PUT /identity/v3/projects/9a64b5dceb3e44fc817c5b6566c89325/groups/9a69d469d489499ba2ba45cb8ddcecbe/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:32.943176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612601d-8d7f-4ed9-b3a5-5db5bc478be5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.943567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612601d-8d7f-4ed9-b3a5-5db5bc478be5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.943930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612601d-8d7f-4ed9-b3a5-5db5bc478be5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.945463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/416] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:32.951423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007760cc-549f-41a5-9743-50197308bf8d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:32.951937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007760cc-549f-41a5-9743-50197308bf8d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:32.951937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007760cc-549f-41a5-9743-50197308bf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:32.970610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-007760cc-549f-41a5-9743-50197308bf8d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:32.972089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-007760cc-549f-41a5-9743-50197308bf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:32.996867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.059243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 203/417] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:33.082243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612601d-8d7f-4ed9-b3a5-5db5bc478be5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.117879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.187326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc8f9-732c-4e37-bd07-1d7f09abfd44 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.187652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc8f9-732c-4e37-bd07-1d7f09abfd44 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.187903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafc8f9-732c-4e37-bd07-1d7f09abfd44 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.189720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 204/418] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:33.194234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612601d-8d7f-4ed9-b3a5-5db5bc478be5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.196934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.197308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.197669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.231303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:33.232797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:33.258980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.259911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.260233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.260490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.290868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:33.299994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:33.301222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:33.302311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:33.308514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/419] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:33.313752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.324892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 205/420] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:33.360889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.476537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.498773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.500916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57215e70-0272-4d7c-a107-aacdd1fc5411 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.514651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f554121-2d58-4335-bf60-98a839b868b2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.552168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.553410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.553828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.554266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.565487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:33.572731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.573424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.573702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.573975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:33.574518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:33.575577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:33.576623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:33.577472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:33.577472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:33.577472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:33.579586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 206/421] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:33.586131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/422] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:33.586999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.593357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.709188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04517b89-350f-42ee-8ae5-fa0d7d3b0b3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.735036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492e9a5-0893-49e4-b6d0-91df096430ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.745779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.746633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.746955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.747321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.748211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:33.748672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:33.750100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef95a5d2-2de4-46e2-a32d-85f1de3dc79e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:33.756320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.761049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 207/423] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:33.762364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c6cc-2e01-4583-aa43-8d9eff728b11 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.762721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c6cc-2e01-4583-aa43-8d9eff728b11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.763025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00c6cc-2e01-4583-aa43-8d9eff728b11 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.765023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 208/424] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:33.766254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00c6cc-2e01-4583-aa43-8d9eff728b11 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:33.779532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.780702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.781142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.781551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.814054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.864134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:33.865808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77048a-9ab2-4952-8f92-a41933d359ec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:33.883527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00c6cc-2e01-4583-aa43-8d9eff728b11 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:33.920987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:33.921959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.922372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.922868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.967731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/425] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:33.969227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac126-1594-4186-8297-790628680af0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:33.969579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac126-1594-4186-8297-790628680af0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:33.969920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bac126-1594-4186-8297-790628680af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:33.986303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-87bac126-1594-4186-8297-790628680af0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:33.987495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87bac126-1594-4186-8297-790628680af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:33.994220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:33.997642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:33.999387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.040590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.059846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/426] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:34.060907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac126-1594-4186-8297-790628680af0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.110869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 209/427] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:34.111972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.160154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bac126-1594-4186-8297-790628680af0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.195791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.196851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.197192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.197531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.198949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.200648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.202494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/428] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.203440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.222507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.226035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb16e46-18f2-4127-8db6-3d5e8f19bebd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.260609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.266135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.267426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.267902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.268413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.280668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:34.288435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:34.289365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:34.290170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 210/429] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:34.290826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.318535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e664fee-77ee-489e-b90c-1408450204e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.355301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.356593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.357060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.357523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] PATH_INFO: `/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.359075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=fbc2051fd3b243d7bc7fd22e9f1b9b6c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.360736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.363266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/430] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/projects/fbc2051fd3b243d7bc7fd22e9f1b9b6c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.364337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.394370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b78d4-f74f-43e6-bb79-371d69dc2b51 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.428531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.429378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.429654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.429924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.431107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.432655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.434673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 211/431] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.435585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.464055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.473879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60f87f-d3fe-45e1-a463-362cc3bd2fa5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.495810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.512271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.513755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.514239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.514699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.515672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:34.516201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.517886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.528157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/432] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.529357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.539822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71550-5221-43f9-8c23-aacc1d14ac6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.574668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.575535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.575814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.576113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.577286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.578832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.580978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 212/433] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.581975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.638812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be8b5-8210-4030-bdb7-4588e8933afd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.678649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.679661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.680162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.680708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.681327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.682595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:34.688220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46b859-ba06-402e-bd2c-07f37fca3653 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.688805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.690569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.698745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/434] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.700118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:34.725316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.725697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.726035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.726310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.754343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.808148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.808148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.845895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582fcf8-b77a-4543-8c4f-331351d1328f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:34.882974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:34.884169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:34.884666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:34.885187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:34.886313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:34.886818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:34.888619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:34.896141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/435] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=fbc2051fd3b243d7bc7fd22e9f1b9b6c&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:34.903032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:34.916219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 213/436] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:34.922711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.057932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da522c1-51bd-4aa8-8a8a-179c4f0937c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.070699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.105620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.106612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.106892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.107214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.109705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:35.111250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:35.113482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/437] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:34 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:35.119413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.242209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4dbeb1-b4ac-46ce-bbe2-12fea3f596b4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.279123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.279983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.280290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.280562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.281744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:35.283202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef6db4-b8b7-48ec-9086-d7ed0419babf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:35.285376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 214/438] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:35.291296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.292242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a8bb9-8ca5-4eee-b124-beea981848c0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.292530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a8bb9-8ca5-4eee-b124-beea981848c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.292807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63a8bb9-8ca5-4eee-b124-beea981848c0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.294380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 215/439] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:35.301201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934592-c347-4921-a450-584f42a53c89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.301480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934592-c347-4921-a450-584f42a53c89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.301752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a934592-c347-4921-a450-584f42a53c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.320672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4a934592-c347-4921-a450-584f42a53c89 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:35.321929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a934592-c347-4921-a450-584f42a53c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:35.389621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.402986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 216/440] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:35.404524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef94d-4c6a-463e-bad7-d3a683b44487 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.404865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef94d-4c6a-463e-bad7-d3a683b44487 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.405141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ef94d-4c6a-463e-bad7-d3a683b44487 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.406440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 217/441] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:35.411910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.412223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.412496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.427698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708a8da-5c26-405c-9d80-495077776f5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.434307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:35.435909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:35.465771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.466436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.466713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.466980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.467811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=491d70ccf49b4888bfe6d30f063be88e, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:35.468143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:35.469611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:35.476547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/442] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=491d70ccf49b4888bfe6d30f063be88e => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:35.477458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.518749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 218/443] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:35.536506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.542916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.618603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94d954-07f8-4dd4-b7c4-6e2a677f2e34 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.661811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.662809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.663199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.663524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.675629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd0129-910a-4f5f-8a11-3b13349868a5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.681106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:35.691661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:35.692598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:35.693727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/444] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:35.698998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.713052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.725111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.725916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.726189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.726886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.739135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:35.747004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:35.748171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:24:35.749042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:35.749643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 219/445] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:35.755405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.819625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33e261-3b8f-4a00-bcc1-73923658c151 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.821524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.864516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084203e-f713-4a69-9744-54becd150765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.865281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084203e-f713-4a69-9744-54becd150765 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.865565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084203e-f713-4a69-9744-54becd150765 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.865842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084203e-f713-4a69-9744-54becd150765 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.866642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084203e-f713-4a69-9744-54becd150765 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:35.867003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084203e-f713-4a69-9744-54becd150765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:35.868442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b084203e-f713-4a69-9744-54becd150765 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:35.877541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/446] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:35.878027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1174d1-a057-4ced-8caf-7ba51034f591 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:35.884071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084203e-f713-4a69-9744-54becd150765 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.913847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:35.914515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:35.914793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:35.915066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:35.915806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:35.916136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:35.917517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:35.926578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 220/447] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:35.932000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:35.976280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:35.983594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084203e-f713-4a69-9744-54becd150765 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.017397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.018050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.018334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.018702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.028353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a2bd1-4d4b-42e2-969f-99b67aa822cd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.049299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.061752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.062392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.062674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.062947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.086287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.087766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.132274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.133658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.177332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.189162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/448] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:24:36.194831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.236579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 221/449] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:36.242214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.256040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.292866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a2981-ffa7-4b4b-90e8-118e1fad9f34 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.326082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.326940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.327216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.327485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] PATH_INFO: `/v3/projects/5a1e8ee61a454b19822b93f53b6d8718` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.328656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=5a1e8ee61a454b19822b93f53b6d8718)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.330097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.332204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/450] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/projects/5a1e8ee61a454b19822b93f53b6d8718 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:36.337408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdfae8-e0de-4cde-b80c-3fe82d332987 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.374213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.375065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.375344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.375611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] PATH_INFO: `/v3/projects/9a64b5dceb3e44fc817c5b6566c89325` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.376809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] RBAC: Authorizing `identity:get_project(project_id=9a64b5dceb3e44fc817c5b6566c89325)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.378370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.380057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 222/451] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/projects/9a64b5dceb3e44fc817c5b6566c89325 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:36.385665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.403579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.439438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccb9b5-08b3-4caa-bf55-e8bb668ac9d1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.470074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.475801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.476704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.476988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.477261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] PATH_INFO: `/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.478461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] RBAC: Authorizing `identity:get_group(group_id=f6b9ac76e6a44beca4a27ad35c92865d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.479929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.481576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/452] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/groups/f6b9ac76e6a44beca4a27ad35c92865d => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:36.487559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.487688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b713bf7-f1ef-4e14-b2d4-7c770077f514 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.522195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.523059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.523338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.523613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] PATH_INFO: `/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.524845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] RBAC: Authorizing `identity:get_group(group_id=9a69d469d489499ba2ba45cb8ddcecbe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.526302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.527935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 223/453] 10.222.0.55 () {56 vars in 1180 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/groups/9a69d469d489499ba2ba45cb8ddcecbe => generated 229 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:36.533565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:36.590548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774eaa4-98bb-4196-a1c7-c18ba8d5c04e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.622859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:36.624993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.625640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.625916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.626183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.626999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] RBAC: Adding query filter params (group.id=f6b9ac76e6a44beca4a27ad35c92865d, role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=5a1e8ee61a454b19822b93f53b6d8718) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:36.627300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.628777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827215ac-1609-4d22-82a5-4a1440890143 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.634603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/454] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/role_assignments?group.id=f6b9ac76e6a44beca4a27ad35c92865d&scope.project.id=5a1e8ee61a454b19822b93f53b6d8718&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:36.641427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd2660d-4760-4287-867b-2e1a02be9f93 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:36.680763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:36.681428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:36.681709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:36.681981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:36.682819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] RBAC: Adding query filter params (group.id=9a69d469d489499ba2ba45cb8ddcecbe, role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=9a64b5dceb3e44fc817c5b6566c89325) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:36.683129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:36.684633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9305-11c6-4f3f-a5da-2f2c46093ce4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:36.693321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 224/455] 10.222.0.55 () {56 vars in 1376 bytes} [Tue May 24 15:24:36 2022] GET /identity/v3/role_assignments?group.id=9a69d469d489499ba2ba45cb8ddcecbe&scope.project.id=9a64b5dceb3e44fc817c5b6566c89325&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:39.461633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ccae-e24c-4e18-a5b4-38bf51916c88 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:39.462162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ccae-e24c-4e18-a5b4-38bf51916c88 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:39.462162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508ccae-e24c-4e18-a5b4-38bf51916c88 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:39.463759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/456] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:39.477700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:39.478534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:39.478949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:39.505015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:39.506443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:39.590359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 225/457] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:39.630961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508ccae-e24c-4e18-a5b4-38bf51916c88 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:39.720048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:39.760396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508ccae-e24c-4e18-a5b4-38bf51916c88 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:39.795804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:39.796632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:39.796907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:39.797192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:39.808314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:39.809784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f51e3177-dd7b-4180-bf21-60515b5ab307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:39.828625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:39.829568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:39.830654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:39.830654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:39.830654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:39.830654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:39.830654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/458] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:39.837204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:39.880300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05567-8bf7-4574-8e03-028e547a8b20 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:39.880623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05567-8bf7-4574-8e03-028e547a8b20 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:39.880847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e05567-8bf7-4574-8e03-028e547a8b20 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:39.882475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/459] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:39.894109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a8be-e69b-4eaf-8eb0-d0cc7f3bf49b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:39.894477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a8be-e69b-4eaf-8eb0-d0cc7f3bf49b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:39.894755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5a8be-e69b-4eaf-8eb0-d0cc7f3bf49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:39.923032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8de5a8be-e69b-4eaf-8eb0-d0cc7f3bf49b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:39.924246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8de5a8be-e69b-4eaf-8eb0-d0cc7f3bf49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:39.956551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:39.979899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7361530-08ab-4671-8ccf-24866ad9aafa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.014859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.015520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.015795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.016101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.016735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/460] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:40.016840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:40.017113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.018511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.019026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf740-77b0-49ea-8203-9b72513422e2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.019516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf740-77b0-49ea-8203-9b72513422e2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.020066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbf740-77b0-49ea-8203-9b72513422e2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.021643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/461] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:40.027393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.027889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.028639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.048420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:40.050133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:40.052919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 226/462] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:40.053630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.100151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.126890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/463] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:40.127790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.175218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db50ce8-008f-4dcf-8756-833bd416ad71 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.211906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.212774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.213006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.213306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.216851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.226426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.227907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.243102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2e6f2-e1fd-4abb-8d86-5ed6968d6b16 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e06cd456-20f1-42bb-8759-b44dcfd567be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:40.245551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:40.246373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:40.247206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:40.247206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:40.247206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:40.247206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:40.247206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 227/464] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:40.247979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97380-a797-479b-b05d-4a5c72eab905 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.248282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97380-a797-479b-b05d-4a5c72eab905 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.248550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d97380-a797-479b-b05d-4a5c72eab905 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.253818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 228/465] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:40.253818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97380-a797-479b-b05d-4a5c72eab905 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.287271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.288223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.288507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.289662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.290376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.291946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.316395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/466] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:40.317223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.359470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.382777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d97380-a797-479b-b05d-4a5c72eab905 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.422274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.423092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.423421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.423708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.436376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.438161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.454630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb520d-c61b-46fb-8dcc-e372d7861ed0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c9372dbc-ba18-425a-92c1-f633ffcf4711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:40.457420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:40.458307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:40.459190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:40.459190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:40.459190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:40.459190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:40.459190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 229/467] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:40.465218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.465540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.465806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.484733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:40.486397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:40.498216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.499812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.500311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.500832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.501927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:40.502442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.504109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.523192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.527533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/468] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:40.528868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.577182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 230/469] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:40.578356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.649847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.662948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a4561d-8048-4ca9-aa7d-b83a7db43ce4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.716739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.717251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f400f-3de8-4e0e-a668-384238f7cd3c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.718001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.718474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.719049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.720768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:40.721271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.722898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.743074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/470] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:40.743074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.759381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.760401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.760734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.761009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.762011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.763704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.799427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 231/471] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:40.799427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:40.830733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:40.909804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1202ee6-0cb5-4cab-a29b-db5d27150bf1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.921515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2c40e-c7d1-48b9-893c-245236dd481c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:40.957628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.958457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.958731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.959000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.960892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:40.961829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.962300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.962759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.963672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.965379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fba31d1-0bad-49ab-8e55-9ff7e4ab1b2d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.971800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:40.973415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:40.995189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 232/472] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:40.995189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94f8d-adc1-437f-baf5-a37abac360fd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.995189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94f8d-adc1-437f-baf5-a37abac360fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:40.995189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94f8d-adc1-437f-baf5-a37abac360fd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:40.995189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 233/473] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:40.999791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:40.999937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.000235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:41.005134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:41.006183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:41.007235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:41.007235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:41.007235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:41.007235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:41.009067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/474] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:41.015233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:41.030645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:41.032305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:41.032724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:41.122272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 234/475] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:41.148803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b277fd-bbaa-43fa-9f30-4b55c27b1db4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:41.155523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:41.176617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:41.202099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:41.203570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:41.204233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.204700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.205662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:41.206274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:41.207885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:41.222620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/476] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:41.228356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:41.288847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb422d6e-b7a4-4c28-bc3a-9fb2e9588f79 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:41.304208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:41.328016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:41.328827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:41.329109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.329483 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.330242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:41.331750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:41.339716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:24:41.339716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 15:24:41.339716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [parameters: {'id': '5c6013dcde5948ec972941571b1055da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 15:24:41.339716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:24:41.348507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbc7b6-87bb-4c7a-b1f0-664ed3dbf595 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() May 24 15:24:41.357426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return ref.to_dict() May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:24:41.358321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:24:41.359300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:24:41.360162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:24:41.361070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:24:41.361882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'id': '5c6013dcde5948ec972941571b1055da', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:41.362689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 15:24:41.363453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.369087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 235/477] 10.222.0.55 () {60 vars in 1131 bytes} [Tue May 24 15:24:41 2022] POST /identity/v3/roles => generated 144 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:41.369087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:41.405265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:41.406226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:41.406506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.406776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.407773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:41.410354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:41.439672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/478] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:41 2022] POST /identity/v3/users => generated 249 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:41.475906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:41.507648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb8abd-0c06-4272-a8f3-5ada8fdb79aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:41.545774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:41.546518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:41.546796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.547121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.560270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7c170dae-494d-42fb-80da-400aad521822 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:41.569223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:41.570175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:41.570983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 236/479] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:24:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:41.577813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:41.693956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613f86f-6a53-47f4-b923-4b8f56a9d841 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:41.716299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:41.731695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:41.732423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:41.732710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:41.732996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:41.733741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:41.734049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:41.735465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc382197-b9ad-4cea-afc9-fc188d886f34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:41.745055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/480] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:42.877695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8e80-8269-4487-8aae-232710657ae7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:42.878446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8e80-8269-4487-8aae-232710657ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:42.878446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8e80-8269-4487-8aae-232710657ae7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:42.879883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 237/481] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:42.886525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:42.886525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:42.887255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:42.912276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:42.913503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:43.009957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/482] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:43.048917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8e80-8269-4487-8aae-232710657ae7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:43.183835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8e80-8269-4487-8aae-232710657ae7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:43.231864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:43.232839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:43.233127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:43.233404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:43.236676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:43.248457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:43.259010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:43.260211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:43.261138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:43.263851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 238/483] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:43.271367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:43.411232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb2540-2071-495d-9419-cf2b47d7889c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:43.431087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:43.460158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:43.460845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:43.461119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:43.461391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:43.462140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:43.462439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:43.463851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53669e-1adf-48d8-b9e6-7cd84cd08e0f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:43.476077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/484] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:43.481597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:43.562522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e4f9-5e60-4776-813b-7cc44449b061 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:43.562760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e4f9-5e60-4776-813b-7cc44449b061 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:43.562988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e4f9-5e60-4776-813b-7cc44449b061 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:43.564514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/485] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:43.569727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:43.570007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:43.570281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:43.594279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:43.595478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:43.642831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ad3563-3392-4aae-8157-05bc0e9edd81 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:43.707320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:43.709789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:43.711153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:43.712865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:43.736302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/486] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:43.737683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:43.739453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:43.768258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:43.769709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:43.770767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:43.770767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:43.770767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:43.770767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:43.772751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 239/487] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:43.778114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:43.797349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:43.893421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:43.955179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2268ae-0571-48ee-83b9-badd13878bf6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:43.959566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:43.987289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244357c-df0c-4982-9706-fe806fe313bb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.024240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.024919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.025193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.025460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.026182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:44.026479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.027937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e35d9-fe49-4d0b-9457-e9a99ad0347b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.035489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.036336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.036615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.036883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.041390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/488] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:44.042860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707abd-d9b8-433e-91a7-ebe0d8c21230 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.043445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707abd-d9b8-433e-91a7-ebe0d8c21230 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.043855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10707abd-d9b8-433e-91a7-ebe0d8c21230 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.045985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/489] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:44.046833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707abd-d9b8-433e-91a7-ebe0d8c21230 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.052749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a2a0cfc1-b514-46cc-be69-115f0d0a65b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:44.060857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:44.061899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:44.062850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:44.067636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 240/490] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:44.069023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.069314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.069593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.087707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:44.089095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:44.164376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 241/491] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:44.164722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10707abd-d9b8-433e-91a7-ebe0d8c21230 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.172716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.203153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.204954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.205450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.205987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.216805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.219225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.227524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:44.242473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:44.243410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:44.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:44.245954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/492] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:44.251012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.284014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.306856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c785a6-e488-41a2-a227-dbf54a7fa53d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.351705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.352915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.352915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.353115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.353757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:44.354061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.355666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.369498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 242/493] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:44.370312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.393439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b11fb-6559-479a-83f4-3a5d25b5b1cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.439099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.440551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.441064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.441541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.457323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:44.468224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:44.469227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:44.470014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:44.473153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/494] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:44.474180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.485498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.489469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24337c3-3fb3-49a9-b7fe-2e7dfa5c8177 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.526706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.527378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.527655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.527921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.528806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:44.529109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.530541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.576132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 243/495] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:44.577128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.577816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.595429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efb1b5-622c-4fa5-b7c8-c1025b70390f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.630262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.631097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.631381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.631656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.642556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.644144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b2fc0f7f-530f-41f0-ae20-84db862d011e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:44.664773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:44.665700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:44.666569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:44.666569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:44.666569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:44.666569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:44.666569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/496] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:44.667274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b71f6-f1cf-4b22-be48-1951c308579e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.667556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b71f6-f1cf-4b22-be48-1951c308579e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.667833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b71f6-f1cf-4b22-be48-1951c308579e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.673653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/497] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:44.674508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b71f6-f1cf-4b22-be48-1951c308579e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.694210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606beea-5466-43ce-bd7b-2ecdda3973a3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.732596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.733261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.733545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.733820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.734573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:44.734884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.736313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.745457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 244/498] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:44.746264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.767749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.780706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b71f6-f1cf-4b22-be48-1951c308579e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.815414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.816070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.816363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.816635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.821477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:44.851489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d5201-2b6b-44bd-8a5c-f50d4d92e541 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:44.885313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:44.885651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.886163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.886503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.886839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.887158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.887686 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:44.887998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:44.889410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26317ae-4709-4452-985a-f01176c5585e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:44.901832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 245/499] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:44.903058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b017-1d63-43f9-850c-fd3d996d6a58 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.903345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b017-1d63-43f9-850c-fd3d996d6a58 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.903620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3b017-1d63-43f9-850c-fd3d996d6a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.919550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3ca3b017-1d63-43f9-850c-fd3d996d6a58 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:44.921084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ca3b017-1d63-43f9-850c-fd3d996d6a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:44.986933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 246/500] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:44.988506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f02d87-8f08-4cca-b345-e7416162e28f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:44.988790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f02d87-8f08-4cca-b345-e7416162e28f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:44.989064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f02d87-8f08-4cca-b345-e7416162e28f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:44.989673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/501] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:44.990462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 247/502] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:44.991021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f02d87-8f08-4cca-b345-e7416162e28f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:44.995215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.004436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.033696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.092327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f02d87-8f08-4cca-b345-e7416162e28f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.098733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3700a7-ddd9-4739-9672-85688c04e083 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.125721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.126593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.126886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.127146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.132413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.133207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.133481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.133746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.134330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.135789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.143924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.145297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-612f3560-75ee-4939-8f90-114e79f70ffc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:45.152080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:45.153052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:45.153997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 248/503] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:45.154722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.154978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.155237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:45.169906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:45.171330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/504] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:45.172481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.174084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:45.175277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:45.236360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.242059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 249/505] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:45.242796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.275249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.288319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d41ef-990a-4977-92e9-031a8e6aba0a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.322876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.323769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.324069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.324345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.335067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.336586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.340539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61afb0c-0151-42a4-826a-bc5b7287f04a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:45.353234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:45.354252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:45.355083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:45.355083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:45.355083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.355083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:45.355083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/506] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:45.355709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.379642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.380544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.380909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.381196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.391378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:45.399233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:45.400153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 250/507] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:45.405636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.453591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d633699e-5f51-4656-b0d2-25dbfe4760a0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.458868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.486124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.486812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.487136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.487408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.488268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:45.488579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.490062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.499753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49f1c2-e46f-4e86-907f-24c411beede2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.514587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.526718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/508] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:45.527533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.536997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.537675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.537947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.538216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.538917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:45.539216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.540641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.557814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 251/509] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:45.557814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.629126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634324b4-65b9-42a6-9afe-4c53dde53b90 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.656370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52ab9a-872e-4f29-a5cd-c6396b96c518 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.662779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.663562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.663874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.664182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.674949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.676484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.679104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.698558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.699233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.699511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.699781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:45.700418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:45.701222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:45.702073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:45.702073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:45.702073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:45.702073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:45.702279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:45.702279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.702279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.702455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/510] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:45.703428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.716101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 252/511] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:45.716919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.765113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.814066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec79b47-1131-4e07-9877-138ac38032e3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.816623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa1db1-d34b-42ac-a518-88169f22a496 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.847691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.848368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.848650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.848922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.852164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.852815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.853110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.853365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:45.854110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:45.854413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.855808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.864117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/512] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:45.864905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:45.903227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.923623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:45.925163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:45.962322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5acb6-bc8e-4491-9932-6c65fe8a8d33 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:45.974541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:45.995630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:45.996450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:45.996731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:45.997000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.004590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.006011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:46.020828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:46.021861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/513] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:46.022927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.028599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 253/514] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:46.029433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.108854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.121469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6e120-7b36-414f-becb-68b8006eb3ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.130408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf24290-686e-4a47-b286-ab5e4fb14c8f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.155314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.155975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.156288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.156565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.157289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:46.157602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.159082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.165331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.166121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.166393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.166659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.171022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/515] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:46.171716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.173573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.175497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:46.189419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:46.190574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 254/516] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:46.191710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.204820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.274031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0dd0a-683a-4211-8eb5-ae40ef02cb45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.294415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b03aad-36ba-4550-a0d6-0b2300f97d63 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.309661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.310468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.310748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.311020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.321362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.322828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.328636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.329309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.329956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.330230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.331677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:46.332009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.333468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:46.340198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:46.341495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/517] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:46.343105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.347694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.356599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 255/518] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:46.357375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.427458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.443445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7af2d3-bb93-4838-9267-47244e783080 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.454617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3b042-abe5-4df8-b2c9-1f9c895d3fe5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.479164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.479965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.480312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.480580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.488160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.488814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.489083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.489349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.490032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.491473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.492320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.493773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:46.510761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:46.511747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:46.512652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:46.512652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:46.512652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:46.512652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:46.512652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/519] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:46.512920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.516412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 256/520] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:24:46 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:24:46.517198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.582729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.633319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a2edd-ba71-4111-af07-bdb559010654 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.638690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5a708-118d-4d81-b645-ccc62bbf6b8c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.675585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.676313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.676594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.676870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.677608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:46.677917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.679296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.679431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.680180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.680658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.681033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.681895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:46.682633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.686198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.693770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 257/521] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:46.694135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.694725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.705050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/522] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:46.706033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.806738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084cf55-bc0e-4a0d-be57-09525e6690b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.821679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.826451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd1ed0-e887-42be-9d2c-0561a6a521a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:46.843015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.843981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.844418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.844821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.863177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:46.863951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:46.864259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:46.864531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:46.865248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:46.865550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.866930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.878831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/523] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:46.879652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:46.918503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:46.920889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:46.940163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:46.988409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e2b23-fbb4-4bcc-9291-1dfd47bd6d17 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.026235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.027131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.027300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.027568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.031575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 258/524] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:47.036070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.036229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.037060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:47.052339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:47.053499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/525] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:47.054751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.057718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.146154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f009a6c-d2b2-4c06-b28a-8559184d79de None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.163544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a87ed-a1fa-400f-8f14-55972824edd2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.182335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.185512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.186573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.186851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.187119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.187943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:47.188292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.189730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.208069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 259/526] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:47.208069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.214715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.215749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.216114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.216448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.224742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.226380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:47.243018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:47.244014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:47.244879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:47.249390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/527] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:47.250464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.312717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.338046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47009616-9db5-4793-86c7-6fc130cd1658 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.368450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cf135-53fa-4e60-8d5f-5076e4772566 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.376146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.377205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.377733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.378174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.390304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.392417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.405633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.406239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.406518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.406791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.407630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:47.407939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.409476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:47.413652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:47.414663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:47.415696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:47.415696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:47.415696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:47.415696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:47.417284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 260/528] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:47.418187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.428820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/529] 10.222.0.55 () {56 vars in 1139 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:47.429863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.464127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.531046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b04417-af4a-4b26-bd1d-9b4a1f4af535 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.535451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1b23a-ed84-4bb4-b4e8-be7d9ff60f2d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.569852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.570566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.570844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.571132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.571847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:47.572194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.573605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.576366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.581425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.582452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.582978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.583471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.584551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:47.585087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.587316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/530] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:47.588225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.592314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.613864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 261/531] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:47.614969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.695065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.701490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb42673-208c-4e3f-85b0-d85ddf473e21 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.732478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547dfaaa-bac3-44d8-aa42-3effff8f9a4b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.739622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.740313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.740592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.740860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.779230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.780317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.780600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.781767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.791291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.793478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:47.817449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:47.818543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 262/532] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:47.819588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.820961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.822709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:47.883086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.916592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:47.926599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/533] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:47.927556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:47.953339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989a34-74e9-46af-95a1-db042cbe2672 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:47.990174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:47.991122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:47.991500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:47.991944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:47.992839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=03707ded8dc84180a4ea90986655b84e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:47.993242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:47.994787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.007962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 263/534] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/role_assignments?user.id=03707ded8dc84180a4ea90986655b84e&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:48.009132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.053652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dcf6-c0b9-4900-8493-22dd446670bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.100083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.102792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.103130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.103408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.105766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:48.106085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.107623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.132646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/535] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:48.140251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.216134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f49cc-82ae-42f9-a194-9919bd22f5da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.249980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:48.256467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:48.283218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.284223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.284570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.284837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.300719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.302589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.320132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347a37f-0ec3-46e6-b2ac-9fcbedc199d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:48.324821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:48.325682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:48.326627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:48.326627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:48.326627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:48.326627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:48.326627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 264/536] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:48.331261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.366439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.367360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.368875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.369304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.371838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=ae9d48a9a9384ada870d31cdbc9c90e3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:48.372326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.375148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.390984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/537] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/role_assignments?user.id=ae9d48a9a9384ada870d31cdbc9c90e3&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:48.505859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5dd37-4551-472a-8d40-03a5b6d60e5b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.561353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.562286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.562729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.563147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.564064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:48.564538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.566538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.584471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 265/538] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:48.589738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.699935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad21fbb-d99d-4983-83e6-96b7ea38d13d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.717351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:48.735144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.735951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.736259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.736534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.743866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.745369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-62ddc4c6-9bbb-419a-89c0-4b2e08a34411 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:48.759970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:48.761044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:48.762074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/539] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:48.768291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.806531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97fe8-2f8f-4f95-8086-9175fedffdfa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.806907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97fe8-2f8f-4f95-8086-9175fedffdfa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.807080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97fe8-2f8f-4f95-8086-9175fedffdfa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.808486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/540] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:48.812433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:48.813688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.813973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.814247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.835445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:48.836687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:48.908246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dff430-02a4-4273-a83d-747d190660d7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:48.947694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:48.948536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:48.948819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:48.949093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:48.949883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:48.950216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:48.952166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:48.961209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/541] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:48.965261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 266/542] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:48.971162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:48.984856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.148482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:49.162422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962b505-07e1-41c8-9166-58f35daaad87 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.167613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be16c07-4377-4e0c-b050-00b676548364 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.187439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:49.231768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.232514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.232800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.233074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.233908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=fbe062ad83f3490282626add7af4947a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:49.234214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:49.235687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:49.247025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.247838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.248139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.248412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.252104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/543] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/role_assignments?user.id=fbe062ad83f3490282626add7af4947a&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:49.261288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:49.269461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:49.270425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:49.271205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:49.275702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 267/544] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:49.277111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.418640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc9457-96d5-4f6f-b50b-8f35ebe499cd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.462833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.463600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.463879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.464173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.464963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:49.465267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:49.466756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:49.481640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/545] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:49.483091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.586181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841927b4-8e2e-4206-8611-1520a44dee6d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.620844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.621688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.621961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.622226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.635368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:49.636866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:49.643625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a40127b9-9072-48e6-b225-a24abd0e9c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:49.655056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:49.655954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:49.656845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:49.656845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:49.656845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:49.656845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:49.656845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 268/546] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:49.665271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.668343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44543f65-e0ea-4cc2-a9b6-511442b3ef3f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.668630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44543f65-e0ea-4cc2-a9b6-511442b3ef3f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.668948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44543f65-e0ea-4cc2-a9b6-511442b3ef3f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.670921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 269/547] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:49.676469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.676749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.677029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.699826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:49.699826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:49.737387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:49.781342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 270/548] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:24:49.805259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4d165-4ef6-45b9-87e7-15977df85e35 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.818776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.839612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.840308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.840583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.840854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.841564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:49.841861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:49.843233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:49.855296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/549] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:49.865442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:49.929115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd2dc8-0cf9-47a6-aba9-269274a36763 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:49.957431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:49.966688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:49.968343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:49.968845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:49.969327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:49.980307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:49.988884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:49.990295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:49.991298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:49.994167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 271/550] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:50.000241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.088761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:50.097091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c464c-7818-4a0d-9c5a-1d2df9d19b7c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.105039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa196e26-d4b3-4a24-b8ea-9cebff154762 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.146013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.147090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.147758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.149288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.150226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:50.150923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.156183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.170546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 272/551] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:50.178134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.186006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.187411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.187883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.188363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.205188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.206736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:50.236659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:50.237663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:50.238744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/552] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:50.243915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.260456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:50.304620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881d69b-c576-4b6c-ae80-a29706224db9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.369872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.371666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.372316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.373866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.397538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.400248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:50.437584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:50.438505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:50.439356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:50.439356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:50.439356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:50.439356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:50.450174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 273/553] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:50.451576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.469511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946584bd-f390-4228-b613-a9dec38028fd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.525313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.526001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.526369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.526540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.528102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:50.528102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.537282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.558716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02f74b-694c-4ca2-b4c3-f6a3c2ae3d88 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.568046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/554] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:50.568753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:50.569961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.604312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:50.635407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.636097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.636383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.636652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.637382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:50.637686 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.639061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.680281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 274/555] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:50.697492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.788909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3c94d-717f-4a96-b4e4-8e9c77bf6bef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.858665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.859328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.859610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.859883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.889351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3491b7-3973-474e-8553-c2ef1f15aa0c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:50.949615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:50.950959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.951249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.951524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.960250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.961739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.969566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:50.971405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5edec2-62d3-4366-905b-1848c1ede9f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c9a03512-6248-4f3f-b383-6fc99a709b9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:50.976215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:50.977974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 275/556] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:50.978903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f943cc-6019-40bd-a2a5-950b7b1f8edd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.979535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f943cc-6019-40bd-a2a5-950b7b1f8edd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.979535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f943cc-6019-40bd-a2a5-950b7b1f8edd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.982498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 276/557] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:50.982498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4ad9-3df5-427e-b7aa-a8af18dff2e5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:50.982498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4ad9-3df5-427e-b7aa-a8af18dff2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:50.982498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf4ad9-3df5-427e-b7aa-a8af18dff2e5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:50.983770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 277/558] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:50.984542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf4ad9-3df5-427e-b7aa-a8af18dff2e5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:50.994610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.016287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.087886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/559] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:51.089340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.089686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.090031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.105501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf4ad9-3df5-427e-b7aa-a8af18dff2e5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.107622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:51.108857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:51.140384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.141044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.141321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.141590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.142521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:51.142823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.144491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d7986-6b3f-4e4e-8361-2a2f977a8053 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.167116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 278/560] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:24:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:51.167116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.167116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.167116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.183110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:51.184159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/561] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:51.184854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.192401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:51.231686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.272375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 279/562] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:51.272375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.288384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.296228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc2f91-cd51-4374-b192-c0aa3e63d9e9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.334638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.335448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.335730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.336280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.346811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.348306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:51.366997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:51.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:51.368844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:51.368844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:51.368844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:51.368844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:51.368844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/563] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:51.369112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.390166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cec0f6-0d43-4ec7-9310-6138dabbb542 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.435250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74295494-757f-44d0-8660-da0c1849192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.436634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74295494-757f-44d0-8660-da0c1849192f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.437085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74295494-757f-44d0-8660-da0c1849192f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.437572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74295494-757f-44d0-8660-da0c1849192f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.450684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74295494-757f-44d0-8660-da0c1849192f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.453050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74295494-757f-44d0-8660-da0c1849192f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.468231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-74295494-757f-44d0-8660-da0c1849192f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:51.472883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:51.473813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:51.474749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:51.474749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:51.474749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:51.474749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:51.474749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 280/564] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:51.475045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74295494-757f-44d0-8660-da0c1849192f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.481934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bef127-7b7d-41bd-9fcc-d650d1ee01b8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.520084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.520892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.521167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.521464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.532547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:51.540665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:51.541625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:51.542464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/565] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:51.543177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.551160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.594130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74295494-757f-44d0-8660-da0c1849192f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.633264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.634067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.634372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.634722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.645287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:51.650333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325e1a8-7051-413e-9f87-7ecb4a7de010 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:51.653203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:51.654125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 281/566] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:51.654988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.701731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.702418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.702693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.702958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.703731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:51.704057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.705573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.718199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/567] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:51.722307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.739574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.769780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24f09-fd35-4e06-aca4-5575709404aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.793304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.807535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.808232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.808517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.808791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.809516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:51.809824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.811208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.824471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 282/568] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:51.829056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.833884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500b809-133c-4ee6-92db-b1a9f3c6e83a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.871667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.872372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.872701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.872980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.873724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:51.874031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.875418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.884385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/569] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:51.885294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.931386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86890-e7f2-4707-8957-ec8a54bb774e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:51.959676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:51.965720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:51.966389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:51.966672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:51.966945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:51.967690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:51.967989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:51.969393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:51.977572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 283/570] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:51.978355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:51.984194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfebe1-6856-4d89-8693-022a436f3b8b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.020054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.020789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.021063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.021332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.024089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.027965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.029474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3e4869b4-7a1b-4795-b691-b26e338afd42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:52.043571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:52.044663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:52.045884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/571] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:52.046535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24dcd5-75ed-45ba-8679-3df27f5a3be2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.046535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24dcd5-75ed-45ba-8679-3df27f5a3be2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.046842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24dcd5-75ed-45ba-8679-3df27f5a3be2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.048215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/572] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:52.048721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24dcd5-75ed-45ba-8679-3df27f5a3be2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.083850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ed8d0-0de9-4a1c-817b-f5638eac2d05 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.116869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.117663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.117944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.118219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.124928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.126387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:52.140593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:52.141650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 284/573] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:52.142576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.156702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24dcd5-75ed-45ba-8679-3df27f5a3be2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.192540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.193317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.193603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.193875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.196545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.204713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.206252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:52.223581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:52.224507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:52.225434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:52.225434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:52.225434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:52.225434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:52.225434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/574] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:52.225719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.246780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d71ad1-54b1-4647-b9a6-e72a42da93d9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.249600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.281433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.282162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.282441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.282713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.293892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.295671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-36962f59-d4bb-4c4c-87ff-ba697fb65e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:52.315009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:52.315926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:52.316845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:52.316845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:52.316845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:52.316845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:52.316845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 285/575] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:52.317807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.318091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.318367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.331216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ad2cb-996c-4c89-88d0-cb487691ac21 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.334332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:52.335510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:52.365366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.366033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.366312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.366586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.367368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:52.367671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.369162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.382255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/576] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:52.383331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.407462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 286/577] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:52.408257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.442764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.500751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.502958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf580c5-6375-42c2-a098-5187ad823e36 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.516706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb9843-e6b2-4ecc-be49-31c05a2a614e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.543290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.543988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.544358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.544696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.545982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:52.546227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.547694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.551124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.551890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.552283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.552625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.553425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:52.553795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.555362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.560444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/578] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:52.561478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.567820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 287/579] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:52.568692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.664279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09873b0-07e2-4bab-a24c-04c76fbcf64a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.675739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78e877-426e-4dcc-9b84-0fc9186db7f7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.676403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.705891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.706832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.707112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.707630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.708794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:52.709109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.711021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.719179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.719179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.719179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.719561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.720448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=eeb122c4f54840a98cd9fc932fb2856c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:52.721291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.722737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.728584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/580] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:52.728584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.729134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 288/581] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/role_assignments?user.id=eeb122c4f54840a98cd9fc932fb2856c&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:52.730540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.744222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.841105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a394f-cea8-45f1-a8a1-ec3f67b5d1a6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.845484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ac50c-8ddc-4320-9057-1cbfbb4111e6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:52.877490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.878396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.878674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.879134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.887297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:52.888382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:52.888517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:52.888679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:52.895864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=c87b5039c28044988e7d8c418c286994, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:52.897653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:52.900865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:52.909217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:52.910393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/582] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:52.911251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.924138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 289/583] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:52 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:52.925338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:52.945262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:52.999653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.027330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b8435-b6cb-451c-82cc-fe5dd9a8e003 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.049097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c190706-c372-4cc6-aada-c057fe2397b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.065436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.066264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.066544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.066818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.075480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.076787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.091457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.092675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.093169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:53.093281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:53.094180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:53.095124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/584] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:53.095729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.096694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.105556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.107374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:53.125842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:53.126850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:24:53.128199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:53.130983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 290/585] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:53.132113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.200906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.218413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1637b5d-e7cc-4749-a213-75b00a94dd2a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.251440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923845c6-2094-4320-bcef-fab29850bc38 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.261116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.261831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.262109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.262816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.263577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:53.263883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.265980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.275781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.282354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/586] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:53.282354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.297653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.298638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.299114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.299593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.300791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:53.301200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.302771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.321338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 291/587] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:53.322405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.414893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbee3c-e35e-454e-a618-01dfbb4e175d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.430892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77100284-8b2e-42f7-90ba-327ff03f7b87 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.454287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.455289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.455489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.455850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.457019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:53.457428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.459244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.476708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.477753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.478173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.478601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.478724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/588] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:53.479668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.490809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.494053 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.495853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:53.519401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:53.520571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:53.521494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:53.521494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:53.521494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.521494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:53.523449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 292/589] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:53.524944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.563022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.593655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1d13-b508-41b4-ae12-c0f2501ff5eb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.629425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.630396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.630737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.631074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.642919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.644408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.651888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce77e10-8f75-4fcc-9ffa-11f1f9dc972e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:53.668749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:53.670204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:53.671152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:53.671152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:53.671152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.671152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:53.671152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/590] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:53.672057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.699373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.700945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.701552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.702024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.715676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.717514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:53.740829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:53.741770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:53.742728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:53.742728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:53.742728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:53.742728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:53.743033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.745770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 293/591] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:53.747308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.786984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8da79-9dae-4374-8390-ff582ad3eb12 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.821952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.822715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.822996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.823265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.824053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:53.824314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.825697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.842323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/592] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:53.843289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.857744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:53.884378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d045ed-4e15-4029-a2e1-20604864dbec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:53.920403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:53.920991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:53.921272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:53.921548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:53.922271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:53.922581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:53.923990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:53.936769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 294/593] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:53.937604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:53.976966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a3f2b3-aad6-4c95-bad8-6c9c18139d9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.005535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.021533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.022208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.022489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.022762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.023489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:54.023794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.025231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.037575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/594] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:54.038402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.058784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3b9d6-766f-47c0-b242-b7ba469ddaaa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.108370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.109572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.110058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.110764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.121025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.122504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:54.137444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:54.138433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:54.139373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 295/595] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:54.139984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.143218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.156908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ef01-7b80-491f-874e-96c7bcf484dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.205883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.207185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.207614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.208050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.217386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.219567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:54.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:54.241217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:54.242321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:54.245351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/596] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:54.246299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.251283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326af754-8974-45e2-8762-1cf185bcde8f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.286990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.287988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.288380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.288745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.296659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.298226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.300625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:54.313684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:54.314849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:54.316023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:54.322032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 296/597] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:54.322974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.364834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15544083-3ad3-4106-a2b8-7ee0f7d9ba8f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.398904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.409671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.410631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.411064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.411481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.412496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:54.412944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.415079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.436840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/598] 10.222.0.55 () {56 vars in 1139 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:54.437957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.441006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8daaad-6c85-419f-89c1-b249c94b6232 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.477022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.477629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.477908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.478179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.478965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:54.479269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.480740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.493081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 297/599] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:54.498598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.541382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f29f88-e842-4f8d-91cc-b384e1a1c05f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.544329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.576600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.577447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.577828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.578198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.579084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:54.579489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.581131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.593731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/600] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:54.594732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.609014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771da9e-54f3-4652-8ce6-e9ec2306a468 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.632693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.646374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.647403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.647888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.648333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.660317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.662044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:54.680602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:54.681655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:54.682562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:54.682562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:54.682562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:54.682562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:54.689415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 298/601] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:54.690979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.698457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb23cc-e541-44cb-9f5d-dcc49fd032fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.731707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.732705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.732988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.733262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/03707ded8dc84180a4ea90986655b84e/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.748128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=03707ded8dc84180a4ea90986655b84e, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.749586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.768193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/602] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:54 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/03707ded8dc84180a4ea90986655b84e/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:54.769992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:54.784813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:54.833465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48242b00-800a-47f0-a8f1-add609495188 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.912222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.913139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.913417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.913687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/ae9d48a9a9384ada870d31cdbc9c90e3/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.932525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9fd77-f934-4d30-bda4-a3a257a62966 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:54.965077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=ae9d48a9a9384ada870d31cdbc9c90e3, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.966567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46e8a3-274c-4fd0-82be-909cb6ad11dc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.972543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:54.973529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:54.973529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:54.973778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:54.974478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:54.974779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:54.976392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:54.991024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/603] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:54.995919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.004545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 299/604] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:54 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/ae9d48a9a9384ada870d31cdbc9c90e3/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:55.028732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cade5-78e9-4c14-97f0-71c556bbc4b5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.029094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cade5-78e9-4c14-97f0-71c556bbc4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.029298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cade5-78e9-4c14-97f0-71c556bbc4b5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.031046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 300/605] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:24:55.037129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.037469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.037829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.066744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:55.068081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:55.096491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.097003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.136512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80517a7c-4041-4a78-85f9-3bf80c930e7c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.178913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.179739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.180013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.180315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.190334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:55.191856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:55.207625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:55.208637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:55.209541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/606] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:55.210033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 301/607] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:55.216067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.261920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.347446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9325840-44be-4bea-b774-4ef3b491ee1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.377375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.387072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.387746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.388040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.388322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.389107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:55.389412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:55.389715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0b1b8-7b5e-48c2-b2b1-26b83652e3ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.391041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce92e5-eb71-4ee2-916d-b524c0b0bc8a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:55.405236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/608] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:55.406588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d2b6-d173-4cbb-bd96-df6e738dd0ba None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.411925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d2b6-d173-4cbb-bd96-df6e738dd0ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.412315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1d2b6-d173-4cbb-bd96-df6e738dd0ba None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.413906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/609] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:55.414623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d2b6-d173-4cbb-bd96-df6e738dd0ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.432328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.434134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.434640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.435138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.449310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-68b892fc-eac6-4234-b96f-0ce58c9de320 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:55.457477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:55.458534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:55.459425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:55.462371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 302/610] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:55.463839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.464355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.464832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.483733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:55.485268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:55.495834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.533414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1d2b6-d173-4cbb-bd96-df6e738dd0ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.557536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 303/611] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:55.558522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.578588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.579804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.580333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.581316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/fbe062ad83f3490282626add7af4947a/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.600438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=fbe062ad83f3490282626add7af4947a, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:55.602707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:55.630724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/612] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:55 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/fbe062ad83f3490282626add7af4947a/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:55.631561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.728296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.740329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af581d77-2591-4338-b93c-d4086e46d481 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.792930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.793537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.793810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.794076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.795142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:55.795442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:55.796871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:55.819643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 304/613] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:55.825668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.836856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.876781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032a6e2-e4fd-442a-9d34-1064797f1168 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.919565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.920985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.921268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.921542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.935888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:55.941818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a840-cd9c-4a9f-88fe-17a4bff41929 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:55.946068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:55.946975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:55.948252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/614] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:55.953773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.016746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.017564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.017898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.018175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.042659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.044220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:56.072464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:56.073825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:56.075074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:56.075074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:56.075074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:56.075074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:56.075294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.079015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f490a-035e-45d8-94c1-e849ce26dfc9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.080095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 305/615] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:56.081739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.122912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.123631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.123908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.124209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.124985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:56.125291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.126760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.136413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/616] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:56.141881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.179874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.224125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fab3c5-8c07-4a6c-a80f-4641e5547e69 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.248427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed124d-f484-40b9-a390-215e002b0d6b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.267062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.268103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.268103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.268416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.269096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:56.269383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.270764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.289814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 306/617] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:56.290265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.291317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.291487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.291748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.296679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.318153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.319638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:56.349534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:56.350474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:56.351360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:56.351360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:56.351360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:56.351360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:56.360177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/618] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:56.365839 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.384352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.445201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566d44a-67b4-4866-ad4c-f7bb493ca858 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.476417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.509046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.510239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.510720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.511190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.522561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.525518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.534024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec2076-f084-4fd9-b704-69657fb7dc4b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:56.555322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:56.556355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:24:56.557314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:56.565815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 307/619] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:56.571488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.606977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.608226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.611239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.611781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.612737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:56.613233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.616283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.638628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/620] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:56.644753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.759983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf014a31-dd8a-4ba1-ab46-ca81fde315f2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.784403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.830263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf14e9e-df96-4ce8-8851-287e18cb507b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.834450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.835172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.835454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.835725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.836603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:56.836910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.838349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.874294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 308/621] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:56.878826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.908570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.910215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.911489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.912632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.930346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:56.931831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.938171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:56.968307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:56.969319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:56.970967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/622] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:56.975301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.040392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f656cd1-ed31-477f-858a-6029202e5a5b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.106169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.106937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.107187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.107457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.137057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.138559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:57.170527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:57.171516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:57.172920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:57.172920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:57.172920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:57.172920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:57.172920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 309/623] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:24:57.179949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.282256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.298919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb115a-4334-476f-bd7f-084473457aab None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.346017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.347178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.347869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.348700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.349857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:57.350314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.352846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f89749-1583-4d3f-8d8b-e2429232950c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.357512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d58f57-b04a-444d-ae7c-e65a4a92ffe0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.373207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 310/624] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:57.373660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced181f7-3c4a-41a8-83c2-d43566188f28 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.373847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced181f7-3c4a-41a8-83c2-d43566188f28 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.374087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced181f7-3c4a-41a8-83c2-d43566188f28 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.375507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 311/625] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:57.376057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced181f7-3c4a-41a8-83c2-d43566188f28 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.400759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.406652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.407352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.407628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.407896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.409133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:57.410033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.412306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a280623-4df0-466f-8825-6f38d6cbf562 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.436136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/626] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:24:57.438609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.439054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.439482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.462002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:57.463938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:57.516235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced181f7-3c4a-41a8-83c2-d43566188f28 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.540319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/627] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:57.542439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.553762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.554732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.555099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.555493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.563180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.564645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:57.579869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:57.581160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:57.582378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:57.583900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 312/628] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:57.588287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.600293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.656759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e57cf9-dfe5-4f90-817a-f4a15445494f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.681134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.698847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.699499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.699773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.700057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.708549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b77fa-b485-4eab-be68-8f3ebd61d56e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.747153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.748048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.748415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.748745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.759399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bc1b5d8e-e641-4f26-afb3-8871540742ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:57.767074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:57.768157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:57.769048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 313/629] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:57.770364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacde89-1f28-4639-aa22-d5a72f3c6a88 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.770364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacde89-1f28-4639-aa22-d5a72f3c6a88 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.770498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacde89-1f28-4639-aa22-d5a72f3c6a88 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.775978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 314/630] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:57.775978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacde89-1f28-4639-aa22-d5a72f3c6a88 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.777422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.778974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.843215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.878757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacde89-1f28-4639-aa22-d5a72f3c6a88 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:57.879178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/631] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:57.880020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:57.902697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:57.918742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:57.919397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.919993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.920292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.921080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:57.921388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:57.922841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf03a8-3604-473a-bd7a-b04c6bc3711d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:57.935583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 315/632] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:57.937352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:57.937639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:57.937912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:57.957083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:57.958528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:57.987341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c582a7c-b8e8-49fb-9331-9a150b78070c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.021156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.021825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.022103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.022373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.023104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:58.023403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.024807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.037092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/633] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:58.037270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 316/634] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:58.037270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.044135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.102432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.136108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.147689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a1c2d-9e47-4094-813b-e7b17f89aac9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.156182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699375b1-aa5d-4bf2-93c6-b39ea010e344 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.183770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.184629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.184906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.185174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/eeb122c4f54840a98cd9fc932fb2856c/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.193957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.194782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.195217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.195489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.201826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=eeb122c4f54840a98cd9fc932fb2856c, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.203743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.206609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.208067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.219280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 317/635] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:24:58 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/eeb122c4f54840a98cd9fc932fb2856c/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:24:58.220466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:58.226526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:58.227532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:58.228856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:58.228856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:58.228856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:58.228856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:58.228856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/636] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:58.229197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.346208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f43719-0c80-4851-a40e-f2c50c1550a4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.348006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9b9b9-d3da-4f74-854d-48749f8ae40c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.358861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.391485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.392406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.392721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.393027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.404871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.405460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.406317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.406439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.406610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.406849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.409637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.424908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:58.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:58.431741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:58.433093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:58.433093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:58.433093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:58.433093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9fea0958-6b19-42bc-a201-b095efffdeee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:58.433697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:58.434592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:58.435539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:58.436214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 318/637] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:58.436214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.436338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/638] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:58.436338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d115-7277-46ee-98f4-ea690624da9b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.436338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d115-7277-46ee-98f4-ea690624da9b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.436338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5d115-7277-46ee-98f4-ea690624da9b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.437566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/639] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:24:58.440984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5d115-7277-46ee-98f4-ea690624da9b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.543099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aadec4-d709-4035-a612-9ed6aa09b4c3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.546559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5d115-7277-46ee-98f4-ea690624da9b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.580352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.581016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.581293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.581561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.581867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.582286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:58.582527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.582628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.582803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.583093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.583776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:58.583949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f02850-6b25-4222-94e8-fef889be2b96 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.584126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.585452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.600390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 319/640] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:58.600390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.600390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.600723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/641] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:58.600797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.600880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.621293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:58.622591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:58.637763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.658185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.700285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 320/642] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:24:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:24:58.701207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.714793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00934bd-dad0-4143-8c2a-00cd2edb5766 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.766957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.766957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.766957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.766957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.767556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:58.767901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.769392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.782042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/643] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:58.782042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.822212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ec12d-0065-487e-9164-6748e53f895f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.865710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.866640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.866794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.867036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.874690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.876238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.882130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21773145-2565-4a15-9fc6-b68fb3ef8073 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:24:58.890271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:58.891205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 321/644] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:58.892499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:58.903347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.905733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:58.923233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:58.924147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:58.924426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:58.924811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:58.932528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:58.933969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:24:58.947807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:58.948799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/645] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:58.949921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.004877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd98a67-0394-495e-8456-a03703cbc30f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.041607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.042504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.042850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.043189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.053596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaf14a-9a60-431a-b963-762d93bcc76b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.054443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:24:59.062230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:59.063368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:24:59.064424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:59.065473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 322/646] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:59.066088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.095176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.096143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.096461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.096712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.108242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.109716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:59.127156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:59.128140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:59.129039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:59.129039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:59.129039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:59.129039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:24:59.129039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/647] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:59.129951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.150327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.160866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.185912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3974ab8-fd53-4561-91df-83d99616d7ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.221495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.222080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.222356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.222625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.223517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:59.223819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.225291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.235793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4bc89-d551-48fb-b80a-82e316ded15b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.238807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 323/648] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:59.239628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.276692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.277378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.277655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.277928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.281342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:59.281342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.281342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.300660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/649] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:59.301498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.353584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9fd13-dd60-4b38-9cd3-41f6058c55a5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.384880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.391793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.392494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.392819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.393096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.393949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:59.394251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.394580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.395627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.404325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 324/650] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:59.409774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.415283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341b32c-a1ce-42fe-a7ec-9847fca4de59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.456217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.456888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.457160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.457740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.458135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:59.458429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.459931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.472334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/651] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:59.472926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.526954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4783f6-bdf6-4b9d-b48a-477bc9b9b26a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.564366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.565057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.565335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.565603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.566431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:24:59.566742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.568198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.574201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 325/652] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:24:59.574853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.583418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c4c918-2566-42c9-be9a-e06277cd73c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.624813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.625622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.625821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.626060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.644770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.662602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.712403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.713898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.726446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5145eb3-2319-4c1f-8037-8475c42d69f8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.770084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.771203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.771592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.772385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.787845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.789607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:24:59.816681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:24:59.817661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:24:59.818617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:24:59.818617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:24:59.818617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:24:59.818617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:24:59.818617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 326/653] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:24:59.818914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.861609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/654] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:24:59.861609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:59.922253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.946293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d513d9-0ffa-4228-b7b5-bbdec2f7eb91 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.956489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:59.983076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:59.983840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:59.983951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b104b3-7a29-4c43-91bf-3874d53b6e2f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:59.984138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:59.984417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:59.993120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:59.994681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:00.010070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:00.011048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:00.011978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 327/655] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:00.012608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.022366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.023047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.023333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.023603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.024372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:00.024675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.026076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.043511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/656] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:00.044479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.121474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57085dbd-3961-40d5-8e22-696fe8c251c9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.154762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b3d31-23d4-4389-b539-5320ab9edfeb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.156707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.157614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.157896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.158166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.166612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.174010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.176718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.197255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:00.197915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:00.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:00.200078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:00.200078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:00.200078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:00.200078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:00.200353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.204067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.204067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.204067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:00.204067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.204407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 328/657] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:00.204515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.204609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.228296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/658] 10.222.0.55 () {56 vars in 1137 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:00.229454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.237882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.330268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5b0a4-6866-43b8-aab3-22143a3fee77 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.353622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7afb80-68e4-43c9-9bce-b552821eb2cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.373628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.374888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.375270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.375642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.387608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.390175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.394798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.395475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.395746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.396015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.396778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:00.397077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.398611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:00.409593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:00.410622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:25:00.411570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:00.415975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/660] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:00.416174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 329/660] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:00.416899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.417132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.446845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.515895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.537721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663949a4-b411-49e9-aef4-9e6ed5a15149 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.538161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e29d557-1299-4a21-8a9c-d55c689e38ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.573762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.574412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.574688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.574951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.575574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.576268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.576550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.576823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.577656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:00.577962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.579396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.597999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 330/661] 10.222.0.55 () {56 vars in 1139 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:00.598642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.647601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.649098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561043f-d17d-453f-9943-2a8e4b9161f0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.676630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.703715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195288a1-c6ba-4c8b-992d-2bd4cbaf2664 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.744828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.745627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.745952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.746172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.753050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.754612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.756450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.762794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/662] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:25:00.764408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981d-1440-4854-b2d1-f7b272733412 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.764790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981d-1440-4854-b2d1-f7b272733412 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.765154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981d-1440-4854-b2d1-f7b272733412 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.766806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/663] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:00.767446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250981d-1440-4854-b2d1-f7b272733412 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:00.773774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:00.774818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 331/664] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:00.780800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:00.888080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250981d-1440-4854-b2d1-f7b272733412 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.888280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade201e-8dc2-4350-991e-65bb9bd7b45a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:00.905656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:00.916842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.917650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.917929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.918210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.920836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:00.921931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.922347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.922752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.929518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:00.930974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6ff55aee-9db8-44c0-8993-7c326c3dbf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:00.948344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:00.949416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:00.950503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:00.950503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:00.950503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:00.950503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:00.956100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 332/665] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:00.956100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:00.956100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:00.956100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:00.970182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:00.971513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:01.001444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.003338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.020981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.048069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 333/666] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:01.048069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.127072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/667] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:25:01.127072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.150273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c14758-9b61-4679-a1f5-f70ea5149318 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.163517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.187502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.188225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.188438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.188736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.189530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.189837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.191308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.207790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 334/668] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.207790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.247711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.254553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f7c548-86cc-4f72-8203-a27c14d90772 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.293135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.293716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.293991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.294261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.295287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.295584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.296987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.311045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2b5be-c5c9-4df4-8832-c67afba2a5e3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.313421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/669] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.314292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.344734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.345530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.345841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.346112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.357016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:01.364779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:01.365726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:01.366575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 335/670] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:01.367865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.409633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.426639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d894f-daac-45dc-b4a3-688ef968eac2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.463665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.465132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.465758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.466221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.473517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.477012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757e365-0112-4a3a-973f-4dc5d6685240 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.480185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.481585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:01.499104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:01.500266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/671] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:01.501445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.518436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.519068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.519344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.519610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.520463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=03707ded8dc84180a4ea90986655b84e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.520763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.522183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.528005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 336/672] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/role_assignments?user.id=03707ded8dc84180a4ea90986655b84e&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.528801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.636085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca59fc-664d-4eba-8c76-4e10015e68e0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.674840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b609da-56e1-485b-b31c-0cd2d221e27e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.686825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.687659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.687938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.688237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.692436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.700301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.701791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.721313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:01.721769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:01.722669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:01.723587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:01.725555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.726140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.726658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.727871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.728202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/673] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:01.728675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.729189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.730692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.746675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 337/674] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.748118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.783242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:01.856225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51628b31-b15c-4839-aa5b-9353afb2f85f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.884115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834607b3-5d58-4500-a5f3-d862017ff4d4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:01.895800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.896833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.897311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.897779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.898699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.899199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.900854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.914151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/675] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.915433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.924596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:01.925266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:01.925540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:01.925809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:01.926598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:01.926900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:01.928375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:01.945891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 338/676] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:01.945891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:01.972468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.020056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fdeadf-636d-4d2e-906d-079cca1a327b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.039558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.056606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.056915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3b2-262a-4f08-9ea0-c1889b8a4e3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.057406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.057680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.057949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.068885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.070339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ca9e3868-1926-4c6b-922b-1107dc4f133f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:02.087716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:02.088657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:02.089548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:02.089548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:02.089548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:02.089548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:02.089548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/677] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:02.090358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a6eb-590b-4b7c-8cb4-82f60c2d5ee4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.090520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a6eb-590b-4b7c-8cb4-82f60c2d5ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.090779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736a6eb-590b-4b7c-8cb4-82f60c2d5ee4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.096455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/678] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:02.096455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a6eb-590b-4b7c-8cb4-82f60c2d5ee4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.099141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.099953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.100269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.100542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.107535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.109031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:02.122746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:02.123840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 339/679] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:02.124807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.210900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736a6eb-590b-4b7c-8cb4-82f60c2d5ee4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.221725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.241676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c90ef7-4217-45bf-ae8a-738a4e83aea3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.246898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.247554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.247827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.248119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.249057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=ae9d48a9a9384ada870d31cdbc9c90e3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.249361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.250809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ddfbf-c23f-47a5-b141-72b2eb12ee13 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.268241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/680] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/role_assignments?user.id=ae9d48a9a9384ada870d31cdbc9c90e3&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:02.269776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.270063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.270336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.279124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.279809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.280124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.280405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.281169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.281476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.282905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.300159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 340/681] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:02.301123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.301422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:02.302976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:02.311127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.399136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/682] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:02.400476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.435789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca79f94-e7f2-4118-b580-c735113ab439 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.481136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.481722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.481999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.482270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.483190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.483508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.485944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.504351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 341/683] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:02.504351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.550729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee9907c-545b-48b2-bb43-8686dd577d31 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.554936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.588002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.596690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.597215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.598080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.598080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.605298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.606758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:02.622841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:02.623817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:02.624826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/684] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:02.625422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.638734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed1a2f3-48ef-480f-ad8b-273d20140826 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.683093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.684545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.685415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.686180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.700939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:02.713331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:02.714317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:02.715201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:02.725523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 342/685] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:02.726219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.758180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a096c-e9bb-41be-b04f-1998b115d23f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.798338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.800352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.800867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.801340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.802471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=fbe062ad83f3490282626add7af4947a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.803083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.804770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.812090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/686] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/role_assignments?user.id=fbe062ad83f3490282626add7af4947a&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:02.813186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.847906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.860172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f04d6-a0f5-417b-a52b-691628b215c2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.900586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:02.909904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.910593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.910886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.911141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.911993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.912327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.913788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.934427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 343/687] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:02.938942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215567-3c67-41c1-b531-7453d8b27db0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:02.942809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:02.977463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:02.978695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:02.979271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:02.979744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:02.980727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:02.981250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:02.982947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:02.999331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/688] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:03.006570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.119074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.145656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d64146-649d-4257-89d3-f4651df8bf59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.162682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7748611-230e-429f-a391-5a3dff417458 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.208138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.209443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.209721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.209993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.221840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.223169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.223864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.224332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.224486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.226080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:03.251919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:03.252975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:03.253882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:03.253882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:03.253882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:03.253882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:03.258851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/689] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:03.259833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.301452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.326406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.327879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.388190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac04389-ede4-4821-bc25-8109ae37e3fb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.424434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.425664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.425951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.426223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.427087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:03.427526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.429160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.441288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.450089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/690] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:03.450268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 344/691] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:25:03.452230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.455664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.554115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.570978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd335c1-abea-43f8-aba6-eccf7df5e031 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.572777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc9abd-38ce-4042-82d5-f063071f0158 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.611960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.613045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.613665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.615390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.623938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.629924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.630477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.630895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.642596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.644455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.646138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.647838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:03.668272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:03.669715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:03.670679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:03.670679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:03.670679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:03.670679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:03.672592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 345/692] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:03.678292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:03.679530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:03.680748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/693] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:03.683821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.684484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.804282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.843314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2709-5fca-4e8e-9936-6cccf194af25 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.854818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6199f5a3-e849-4165-b93c-48316ae4eba4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:03.907053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.909285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.909730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.910596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.911913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:03.913196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:03.913664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.915277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.916093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:03.916509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:03.916914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:03.917770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:03.920125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:03.921616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:03.928755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:03.947178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 346/694] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:03.951310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/695] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:03.956776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:03.958592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.071371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fcf0-9085-4e79-b5ee-65879fbd10c3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.075760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecffba6-70b3-4ac9-a566-c941f597923b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.093686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.110372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.111752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.112200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.112573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.112822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.113355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.113723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.114064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.120974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.122603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6ffdd8bd-ce4e-4467-8211-f28f76218571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:04.138031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:04.139155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:04.140240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:04.142361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 347/696] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:04.143791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42d0b-b7f8-42fd-839d-72396df0870c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.144240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42d0b-b7f8-42fd-839d-72396df0870c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.144681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a42d0b-b7f8-42fd-839d-72396df0870c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.146328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 348/697] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:04.151883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a42d0b-b7f8-42fd-839d-72396df0870c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.196076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.197522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719d5519-8cde-40ca-acfb-28724936750d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.213477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.260355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a42d0b-b7f8-42fd-839d-72396df0870c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.296510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.297267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.297547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.297816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.298695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:04.299059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.300647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.305980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/698] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:25:04.307559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.307996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.308536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.320524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 349/699] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:04.328255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.333734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:04.335106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:04.349565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.411139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/700] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:04.412399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.442521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482055-8b16-4713-9b67-4c65ef29497e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.461968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.481221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.482116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.482450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.482721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.499234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.500731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:04.518452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:04.519448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:04.520553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:04.520553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:04.520553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:04.520553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:04.520553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 350/701] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:04.525338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.537726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5c1a2-c168-470f-941e-c7726921aaf5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.573997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.575014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.575476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.575926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.576980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=eeb122c4f54840a98cd9fc932fb2856c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:04.577480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.579275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.587201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/702] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/role_assignments?user.id=eeb122c4f54840a98cd9fc932fb2856c&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:04.588496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.625333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.692760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57ba9d-35bf-4677-8395-9a5f4b5aa88f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.768651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.771263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.771549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.771823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.772525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.778032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.797718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8b904-639b-4ac8-a006-bc0a4184ddda None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-da5d5e2f-41f7-4ffa-81c8-94b3f460770b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:04.800685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:04.801639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:04.802478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:04.806773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 351/703] 10.222.0.55 () {56 vars in 1129 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:04.808499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f595f6-2dce-448d-aadf-ed9a4e74652a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.808925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f595f6-2dce-448d-aadf-ed9a4e74652a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.809335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f595f6-2dce-448d-aadf-ed9a4e74652a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.810973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 352/704] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:04.814819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f595f6-2dce-448d-aadf-ed9a4e74652a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.851391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.852047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.854432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.854432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.854432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:04.854432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.854738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.855663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061174e-a3b2-40ec-98a2-77cdef582eea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.870856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/705] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:04.872481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.872768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.873040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.892128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:04.893745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:04.947401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f595f6-2dce-448d-aadf-ed9a4e74652a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.986062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/706] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:04.987608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.993081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:04.994718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.995581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.995905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.996330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.997186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:04.997488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.001492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.011619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 353/707] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:05.017846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.137716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:05.140701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d157969-714f-41bd-b380-9f89ed7a5820 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.145468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28b091-6036-4727-b490-1bbdf4720b38 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.182319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.183286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.183659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.184163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.184907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:05.185248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.186729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.189651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.191387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.191822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.192270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.200054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 354/708] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:05.200054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.215488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.217040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:05.239002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:05.239995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:05.240970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:05.243352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/709] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:05.249431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.368491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:05.423449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7f543-b37b-4bcc-bf60-5b39fb008249 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.442342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74896bc1-d96e-408a-acea-e674ba77d0fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.467942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.468788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.469322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.469698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.471657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:05.471657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.473715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.480998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.481799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.482075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.482344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.495217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/710] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:05.496922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.497169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.498643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:05.520777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:05.521757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:05.522671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:05.522671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:05.522671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:05.522671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:05.522671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 355/711] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:05.529696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.556450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:05.634391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d27c9-3b7a-40a2-a193-57bbc3a664c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.660661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:05.675278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9e58f-7c32-41e2-8bdb-aaed146ef6db None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.678635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.679300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.679580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.679853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.680731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:05.681040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.682493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da65446-5c3b-456b-93fe-a6b2cc178e0d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.694330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/712] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:05.695871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41c69-e8a5-44e0-a0dd-603420a1534b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.696305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41c69-e8a5-44e0-a0dd-603420a1534b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.696587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce41c69-e8a5-44e0-a0dd-603420a1534b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.702418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/713] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:05.703912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.704247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.704523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.733145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.733839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.734116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.734385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.735123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:05.735424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.737337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:05.739857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:05.741114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:05.764972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 356/714] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:05.777145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.867920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/715] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:25:05.909701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:05.910672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:05.944919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c3e38-bc7f-44ea-a4b1-30dbc728098c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:05.985038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:05.985725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:05.986007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:05.986364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:05.987446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:05.989232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:06.036183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 357/716] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:25:05 2022] POST /identity/v3/users => generated 254 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:06.102608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df195008-d800-4d04-ac15-138d00b50b55 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:06.121843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:06.148856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:06.149738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:06.150021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:06.150291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:06.150958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:06.152386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:06.171138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:06.172207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:06.173089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/717] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:05 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:06.178568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:06.304205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dce4fd-2b74-4fd7-a0a3-edc5b847cf51 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:06.333167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:06.345964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:06.346703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:06.346989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:06.347263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:06.347989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:06.348328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:06.349790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca82744-6a88-4214-9edb-3a85a1c9290a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:06.355294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 358/718] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:06.360506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:06.494788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba24d10-4b4d-417e-906d-50fca9db1862 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:06.534507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:06.535246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:06.535533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:06.535810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:06.536585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:06.536898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:06.538302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a07c6-cc7d-4d9e-8023-798de4856b61 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:06.545240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/719] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:07.203388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6dfb1-a95b-4b06-be77-1ceea58b8c82 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.203953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6dfb1-a95b-4b06-be77-1ceea58b8c82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.203953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6dfb1-a95b-4b06-be77-1ceea58b8c82 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.205712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 359/720] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:07.217463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.217950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.218085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.235748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:07.237034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:07.285506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:07.322712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/721] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:07.364410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6dfb1-a95b-4b06-be77-1ceea58b8c82 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:07.503887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6dfb1-a95b-4b06-be77-1ceea58b8c82 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:07.545449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:07.546635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.547121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.547660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.559206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9fbdfc9a-e56a-415c-8de1-e1f68daa6b4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:07.572412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:07.573475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:07.574349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:07.576832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 360/722] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:07.583033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:07.752120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e2534-7c4c-436a-b220-648bd65e78f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:07.795878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb1f7-5b90-4ab0-92b0-d179d15fc10d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.796521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb1f7-5b90-4ab0-92b0-d179d15fc10d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.796996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41eb1f7-5b90-4ab0-92b0-d179d15fc10d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.799042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 361/723] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:07.803748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.804048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.804344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.807172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:07.807844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:07.808143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:07.808410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:07.809272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:07.809571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:07.811199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:07.821637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:07.822970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:07.824380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/724] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:07.829691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:07.890442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:07.893380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:07.929443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 362/725] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:25:07.971307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.028469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f509e-48a8-4307-b12a-a7fa4158eded None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.097212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.097942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.098221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.098496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.119876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.121390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8a90db3a-5b6a-41ed-8654-89d762e70b17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:08.147099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:08.148265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:08.149178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:08.149178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:08.149178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:08.149178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:08.149178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/726] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:08.150087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dedb-9f91-4d8c-b8dc-26bf1a97b20f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.150371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dedb-9f91-4d8c-b8dc-26bf1a97b20f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.150643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33dedb-9f91-4d8c-b8dc-26bf1a97b20f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.156022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/727] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:08.156022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33dedb-9f91-4d8c-b8dc-26bf1a97b20f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.159278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0dd6e-218f-49bd-8596-fb72d7184c1b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.206007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b10058-269d-44db-a969-b086360f543c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.206683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b10058-269d-44db-a969-b086360f543c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.207129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b10058-269d-44db-a969-b086360f543c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.207582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b10058-269d-44db-a969-b086360f543c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.208555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b10058-269d-44db-a969-b086360f543c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.210176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b10058-269d-44db-a969-b086360f543c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.240089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 363/728] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:07 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:25:08.240089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.240089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.240089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.277032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:08.278241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:08.283939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:08.317304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33dedb-9f91-4d8c-b8dc-26bf1a97b20f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.356043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:08.392070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.392785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.393060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.393396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.394129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:08.394429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.395820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.413906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 364/729] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:08 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:08.418475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/730] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:08.424907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.451850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.566593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785dd8e-cc2c-4ad8-874c-2a36e393d98f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.578041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a341a9-d3eb-4463-9c25-2b256c960a73 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.603927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.604952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.605231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.605502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.616984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:08.623126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.624196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.624675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.625163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:08.625681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:08.626577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:08.627530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:08.628195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/731] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:08.633185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.636885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.639241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:08.641045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:08.661165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:08.662195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:08.663150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 365/732] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:08.669175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.709428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:08.784999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8076d-f148-46f8-9c2d-d6ab90105016 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.788384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b2d3-e94f-47eb-b917-b04835abb55c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:08.826507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.827667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.828317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.828802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.829799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:08.830511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.832136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.838972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:08.839669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:08.839947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:08.840248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:08.841388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:08.841693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:08.843134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:08.843892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/733] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:08.850054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.876950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 366/734] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:08.884520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:08.985227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.004415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.008141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de49e-059d-4d0f-9a14-503586a53fe7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.061215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7eb77-b331-4181-b9df-9f3b76841627 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.087607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.088322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.088603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.088878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.101239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.102065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.102338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.102607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.128435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.130113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:09.163419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:09.164401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:09.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:09.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:09.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:09.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:09.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/735] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:09.170689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.240646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.243482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdf45a-9a1f-4f38-bbfc-241b850acb80 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.352290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.356826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67770121-f2a4-4ee0-9212-2b59350c1e83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.364352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 367/736] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:08 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:25:09.365997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057e75-c9eb-49ac-a3df-ae9c59a123a9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.366438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057e75-c9eb-49ac-a3df-ae9c59a123a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.366812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057e75-c9eb-49ac-a3df-ae9c59a123a9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.372582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 368/737] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:09.373465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057e75-c9eb-49ac-a3df-ae9c59a123a9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.377099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.397438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.398026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.398302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.398570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.399294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:09.399595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.401006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8669db-8ad8-429b-ba0f-95785d957892 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.417391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/738] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:09.417391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb06a8-f373-44f1-9a04-c480131148be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.417391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb06a8-f373-44f1-9a04-c480131148be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.417391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb06a8-f373-44f1-9a04-c480131148be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.432788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-91eb06a8-f373-44f1-9a04-c480131148be None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:09.433989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-91eb06a8-f373-44f1-9a04-c480131148be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:09.487685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057e75-c9eb-49ac-a3df-ae9c59a123a9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.506050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/739] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:09.507372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57287-3658-48e7-bf45-1255107a5d3b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.507656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57287-3658-48e7-bf45-1255107a5d3b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.507927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b57287-3658-48e7-bf45-1255107a5d3b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.512070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/740] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:09.512070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57287-3658-48e7-bf45-1255107a5d3b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.533760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.534896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.535329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.535601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.546774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.548264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-aaa3cad4-83b7-4a89-a819-ee02a67294d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:09.568232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:09.569131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:09.570024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:09.570024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:09.570024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:09.570024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:09.572164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 369/741] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:09.574261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.574545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.574962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.592140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:09.593336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:09.621077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b57287-3658-48e7-bf45-1255107a5d3b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.630050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.632334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.659515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.660276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.660547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.660816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.662503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 370/742] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:25:09.663969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.682634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.686375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:09.704516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:09.705724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/743] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:09.711906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.780402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca19c8b-0540-4619-9936-0edd64687d18 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.815092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766d93a3-20ea-40bd-80c1-4dde98a0bd43 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.816097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.817001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.818076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.818076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.827860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:09.835597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:09.836604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:09.837465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 371/744] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:09.838599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.856942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.857708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.857987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.858259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.858968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:09.859268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.860780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:09.869402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.878544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/745] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:09.878544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:09.908243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:09.948136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f37a4-13ab-467e-80b1-fd02b9507ad9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.980213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37bb09-b8de-4bf0-a004-ead29739de3f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:09.982018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:09.983405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:09.983682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:09.983950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:09.984704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:09.986168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.000750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 372/746] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:09 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:10.001529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.032322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.033219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.033501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.033771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.034705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.035636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.036687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.063495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/747] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.063495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.122535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.136662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed2509-8855-48ab-9bd1-a77abb0f70d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.181050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.181996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.182485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.182939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.183899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.184379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.185943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.186414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.196224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 373/748] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.197831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.205017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13185f72-f03b-4ce0-b7f8-4a6f6cee289a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.244636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.245445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.245724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.245990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.254014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.255492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:10.269908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:10.270872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/749] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:10.272141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.321492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebda586-9b60-4dc2-8009-404588379792 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.358315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.359120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.359397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.359667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.373544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.375072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.389636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c207d-19c7-41ca-abb5-c1f029539c20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:10.395135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:10.396059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:10.397128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:10.397128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:10.397128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:10.397128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:10.397128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 374/750] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:10.397426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.406646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.434249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.435523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.435983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.436457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.448952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.450904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.453040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:10.471975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:10.472936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:10.473869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:10.473869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:10.473869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:10.473869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:10.476069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/751] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:10.477086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.504925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf6da1f-e423-4c0d-84e7-1d6425e3913b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.541013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.541859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.542238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.542608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.543595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.544534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.545993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.559317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 375/752] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.560305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.593010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797fc1-c181-462d-92e1-6e0d27907558 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.632817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.636877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.637554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.637875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.638198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.638914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.639217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.640645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.653952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/753] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.654804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.682270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4acbb-36f2-4169-8be8-9dbf14144dd4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.723390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.724327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.724813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.725261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.726210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.726689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.728362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.734436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.745562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 376/754] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.746543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.783981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82fcaf4-3a18-4e6b-bf4f-95eeb7304591 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.819476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.820510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.820975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.821473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.822586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=eeb122c4f54840a98cd9fc932fb2856c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:10.823037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.824869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.831255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/755] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/role_assignments?user.id=eeb122c4f54840a98cd9fc932fb2856c&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:10.832392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.871238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50133a-a762-4c2e-9c81-5c6c1412debc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.912824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:10.926697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:10.927830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.928296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.928710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.939544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:10.941694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:10.959755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc462c91-ee3b-47a3-bb9b-f0db523620b0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-71340488-d78d-4b5f-bbfd-afde792d354b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:10.963378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:10.964484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:10.965666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:10.967964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 377/756] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:10.969617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a36f6-bd12-461a-ae3e-edc395b1f755 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:10.970055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a36f6-bd12-461a-ae3e-edc395b1f755 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:10.970509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a36f6-bd12-461a-ae3e-edc395b1f755 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:10.973307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 378/757] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:10.974870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a36f6-bd12-461a-ae3e-edc395b1f755 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:10.983906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.009808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.010794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.011267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.011654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.024238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.025879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d8083efd-09d4-49dd-a1be-2ee567a9661f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:11.041521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:11.042543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:11.043581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/758] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:11.044710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.044995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.045263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.062527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:11.063694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:11.111235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a36f6-bd12-461a-ae3e-edc395b1f755 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.156165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/759] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:11.157075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.157379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.158357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.158778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.159481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.161420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:11.161900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.163558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.178623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 379/760] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:11.184362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.234009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.290546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25fe5c-4b04-4bb9-a837-919c73498ce4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.302498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.307423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4cab0-39a1-414a-96e2-ae99f9e03b87 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.330271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.331263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.331681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.332153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.333124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:11.333567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.335150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.350099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.351006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.351280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.351550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.354919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=c87b5039c28044988e7d8c418c286994, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.356411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.358633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/761] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:11.359680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.382576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 380/762] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:11 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:25:11.384050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.472940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb2553-2d95-472c-ade6-79c83505bd4a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.537771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.539129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.541060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.542222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.543428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.546167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.554449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcea21a-7a6f-44f9-91df-4d6183422398 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.570066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:11.571740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:11.573091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:11.573922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/763] 10.222.0.55 () {56 vars in 1129 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:11.585116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.615509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.616628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.617139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.617618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.659910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.706460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.707950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.745483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea26be1-cf5a-42d8-a69e-61c8913159da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.790726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.792294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.792770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.793339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.794552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:11.795044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.796752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.802681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/764] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:11.809452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.813893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 381/765] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:11.814423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:11.828451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.915015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3dca-ceea-46a4-89a4-4a94eb544017 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:11.953336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:11.954135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:11.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:11.954687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:11.973732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:11.975206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:11.986218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:11.996098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb889b1f-0e89-4a18-94fd-0fc93b196a76 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:12.000561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:12.001527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:12.002456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:12.002456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:12.002456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:12.002456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:12.002456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 382/766] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:12.013235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.050946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440b63-046d-40df-9013-366f799725bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:12.052684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440b63-046d-40df-9013-366f799725bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.052968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440b63-046d-40df-9013-366f799725bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.053244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a440b63-046d-40df-9013-366f799725bf None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.053970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440b63-046d-40df-9013-366f799725bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:12.054277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440b63-046d-40df-9013-366f799725bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:12.060097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a440b63-046d-40df-9013-366f799725bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:12.084102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/767] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:12.087445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440b63-046d-40df-9013-366f799725bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.160584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:12.242511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00369d06-8037-46ba-87ef-0ff8cd331770 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:12.309673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a440b63-046d-40df-9013-366f799725bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:12.320961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:12.321643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.321924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.322197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.322941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:12.323243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:12.324654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:12.345172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 383/768] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:12.349917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.379382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:12.380136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.380418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.380691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.381537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] RBAC: Adding query filter params (interface=public, service_id=4ddfe4a7f07e4554a3234d88c68653fe, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:12.381849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:12.383253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:12.403366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/769] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/endpoints?service_id=4ddfe4a7f07e4554a3234d88c68653fe&interface=public®ion_id=RegionOne => generated 194 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:12.407569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.448538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:12.538050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10455c99-e743-4533-97b5-61200ca8fdfd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:12.550862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12257ba-7020-44c0-ac4c-5908c7e7c06b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:12.564862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:12.587150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:12.588929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.589537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.590027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.600227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:12.602015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:12.603450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:12.604162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.604444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.604715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.605701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:12.607541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e3e31-2dc2-4904-9cd0-2d73f10dfbb6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:12.620071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/770] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:12.620071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08aad1-262f-4569-89a4-70e7b67bbea5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.620071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08aad1-262f-4569-89a4-70e7b67bbea5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.620071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08aad1-262f-4569-89a4-70e7b67bbea5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.621546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/771] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:12.623021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:12.623500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:12.624149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:12.629498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:12.630474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:12.634898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:12.645610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 384/772] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:12.645610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.682110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:12.682110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:12.812787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/773] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:12.847062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:12.874239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:12.998106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0118a0d-6930-403b-8ab1-b69ed23aaab8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.014849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0c52-cf01-4588-b081-3e3cc4e05613 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.055820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.056911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.057193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.057703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.058359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.060494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.066209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.067160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.067444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.068060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.072071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:13.072071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.072071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:13.075798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:13.076805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:13.077646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:13.079073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/774] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:13.085630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.086392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 385/775] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:13.092225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.175943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:13.216896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:13.241809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e019c8-f8f9-4508-8b75-1f9a942505a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.309803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.311081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.312853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.313360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.315362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:13.316474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.318097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.328918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/776] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:13.336475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.351689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d68c3a7-3847-4ce6-bf11-2993bbf21f89 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.414647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.415310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.415589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.415861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.416724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=69a0a21790ed496ba0e1e397a9daea09) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:13.417032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.418478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d25f42-127b-481b-adfd-0712703f1f01 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.429012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 386/777] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/role_assignments?user.id=69a0a21790ed496ba0e1e397a9daea09&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:13.467942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e1d-9fbb-424c-affa-efe596d833a2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.468164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e1d-9fbb-424c-affa-efe596d833a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.468441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78e1d-9fbb-424c-affa-efe596d833a2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.469958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 387/778] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:13.477612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.477947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.478160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.497439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e522f6-8d6e-49a9-85ac-3c5b372764c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.512129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:13.513348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:13.565706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.566393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.566661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.566930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.567708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:13.568013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.569507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.579964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/779] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:13.585261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.604404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:13.636213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 388/780] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:13.653847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.704162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:13.791478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ebcb7-48e0-4f6e-bd20-fd67c7cb0b39 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.796294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7e1d-dd35-4330-b828-0637835b6f80 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:13.837944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.839690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.840754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.841315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.858353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:13.861313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:13.861741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:13.862247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:13.863426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] RBAC: Adding query filter params (interface=public, service_id=a8642627ea9a4975ac388975d73e53ec, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:13.864104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:13.868449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:13.869878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:13.872329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:13.873431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:13.874421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:13.881501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 389/781] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:13.882951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:13.890082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/782] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/endpoints?service_id=a8642627ea9a4975ac388975d73e53ec&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:13.895463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.038812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.055394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98039ba1-24da-4d93-9edb-cf1f736274ee None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.056409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.092561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3912b03-4a0c-4024-b547-6b15c07645a9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.124249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.124930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.125208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.125481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.126231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:14.126706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.128081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.149170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 390/783] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:14.155631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.165718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.166704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.167165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.167613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.168506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.170419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b3677-fe62-4d38-9e5c-38c9bf8fc13b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.184602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/784] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:13 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:14.274907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366b65-cddb-41e8-a35d-a01abbb85baf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.275488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366b65-cddb-41e8-a35d-a01abbb85baf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.275968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65366b65-cddb-41e8-a35d-a01abbb85baf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.277613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/785] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:14.284647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.285125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.285478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.295643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d87ed-70b6-4e10-9b41-650503a22719 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.315455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:14.318576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:14.342919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.344109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.344560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.345030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.357161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.358909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:14.383549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:14.385205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:14.386122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:14.386122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:14.386122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:14.386122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:14.389558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 391/786] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:14.391833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.404984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.426028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/787] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:14.450372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.514062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631173e8-0ec5-4400-9d6a-10d52e9c2593 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.555215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30544461-1ade-47cc-a5c8-fbcb01691fb6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.562106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.562870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.563212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.563547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.564408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.564547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:14.564988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.567033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.584373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 392/788] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:14.585905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.597551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.598364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.598642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.598904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.609626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:14.625841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:14.626894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:14.627732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:14.628446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/789] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:14.636942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.688487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.757864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c0865f-31f4-49c9-93e4-282669b2ecc9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.767334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ef710-2382-4693-bb08-7ec1a297c394 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:14.801940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.808292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.808831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.809259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.809744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.810647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:14.811102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.812735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.833083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/790] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:14.838071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:14.839109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:14.842091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:14.842724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:14.843034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:14.852819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:14.854633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:14.871663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:14.872706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:14.873658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:14.880517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 393/791] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:14.882428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.041486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3823cf9-8856-4b35-9d06-72ff18a3df96 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.063730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97901a97-3937-4774-8656-05b4dc5833bf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.078698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.079648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.080159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.080448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.091312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.093403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:15.112263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:15.113207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:15.114106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:15.114106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:15.114106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:15.114106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:15.114349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.114902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/792] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:15.120431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.139345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.140071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.140346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.140615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.141465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:15.141769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.143282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.172571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 394/793] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:25:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:15.178035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.183849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.246384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b137d5-20ba-4cb2-87fe-bd142f52bf66 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.282066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.283411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.283875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.284339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.285359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:15.285840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.287430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.301251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/794] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:15.313586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.487921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933c4f8-9376-4e6c-83c2-d8ba28aa4c8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.500896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.505501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81fc49-7799-47c7-981c-c3152a8b8397 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.531330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.532647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.532985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.533255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.547521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.548000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.549052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.549738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.550198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.550660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.558444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.560102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b686999e-bdf2-4977-b06c-74bd63d8acc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:15.569692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:15.570825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:15.571934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:15.571934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:15.571934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:15.571934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:15.572985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 395/795] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:15.574331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c203f3f-2134-401a-9922-5f4a2502491f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.574673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c203f3f-2134-401a-9922-5f4a2502491f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.575007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c203f3f-2134-401a-9922-5f4a2502491f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.580654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 396/796] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:15.580654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c203f3f-2134-401a-9922-5f4a2502491f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-782e6452-6bd7-43c4-85ee-6776ce0c2548 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:15.582940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:15.583872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:15.584852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/797] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:15.585749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.585984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.586266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.608813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:15.611056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:15.622759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.696061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/798] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:15.696061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.726333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c203f3f-2134-401a-9922-5f4a2502491f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.774398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.775692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.777130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.777590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.778505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:15.778995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.780570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.799065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.799541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 397/799] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:15.802056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.810866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a15d3-7c8e-48fe-8dc9-58a6f8648273 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.846485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.847156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.847431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.847745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.848603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:15.848904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.850352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.873742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/800] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:15.873742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:15.933399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:15.940525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c3313-5940-455e-9719-87bd48c4c79e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:15.977637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:15.978750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:15.979044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:15.979405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:15.980217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:15.981627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:15.994030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:15.995204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:25:15.996273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:16.003231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 398/801] 10.222.0.55 () {56 vars in 1129 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:16.003231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.011683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa268a-cb58-44a4-99d9-b2999115a4a4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.056854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.058174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.058625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.059048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.068514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.070443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:16.085015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:16.085967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:16.086852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/802] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:16.087975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.099580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.111952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f91671-1192-4ccb-b425-a7a401c3cd66 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.147343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.147989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.148291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.148561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.178564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.194901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68fb24e-389a-4950-ad9c-8798e83c2ddd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.222349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.223901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.230296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.230963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.231237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.231893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.232626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:16.232931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.234639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442111a4-90d7-42b9-b4fd-16444665360b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/803] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420a840-9b5a-404b-a0a4-d9c4986d30d5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420a840-9b5a-404b-a0a4-d9c4986d30d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420a840-9b5a-404b-a0a4-d9c4986d30d5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/804] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:16.243986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420a840-9b5a-404b-a0a4-d9c4986d30d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.316536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.334937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 399/805] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:16.334937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.354254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420a840-9b5a-404b-a0a4-d9c4986d30d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.388892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.389547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.389827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.390105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.391370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:16.391481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.391564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.392805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5860df00-f95a-4b6c-a8ec-ecf370abd494 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.405460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/806] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:16.406794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.407076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.407346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.424055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:16.425228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:16.439324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec21c47-4223-4bb2-845a-9239ac012f60 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.479065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.480006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.480538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.480805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.481498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:16.481799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.483156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.492748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 400/807] 10.222.0.55 () {56 vars in 1124 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:16.492748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.506059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/808] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:16.506762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.570789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.611383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17250d9-07b1-4d0b-a5de-685beb590b53 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.614820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed500d3-b1e1-4ed9-b026-f94be47d0a09 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.637570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.649221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.650014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.650284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.650545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.651951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.652886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.653172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.653446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/eeb122c4f54840a98cd9fc932fb2856c/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.655403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=eeb122c4f54840a98cd9fc932fb2856c, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.656882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.662800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.664493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.672927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/809] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:16 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/eeb122c4f54840a98cd9fc932fb2856c/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:25:16.673786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:16.692623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:16.693624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:16.694537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:16.694537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:16.694537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:16.694537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:16.698751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 401/810] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:16.699607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.806755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e226740-158b-42e8-9b5d-6456886d2c69 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.810230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81092050-a17e-46bd-94c0-3067d7bf9b0f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:16.833975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:16.848370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.848632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:16.849042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.849145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.849297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.849450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.849596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.849697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.850380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:16.852110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf1d215-3ded-4af8-89d5-8505a8d79df1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:16.862403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:16.871218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:16.872343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:16.873197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 402/811] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:16.873893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.885345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/812] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:16 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:16.885546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbf29-3e99-43ab-bdb8-d6c4292b30b1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:16.885867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbf29-3e99-43ab-bdb8-d6c4292b30b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:16.886145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbf29-3e99-43ab-bdb8-d6c4292b30b1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:16.891606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/813] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:16.891606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcbf29-3e99-43ab-bdb8-d6c4292b30b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:16.930005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.006258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7685ae-4a8d-4ac3-8236-0adff350be32 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.045587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcbf29-3e99-43ab-bdb8-d6c4292b30b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.052192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.052874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.053182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.053464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.054196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.054498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.056732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2194f01-4cfb-4c7c-a68f-1bfa4e56124d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.075032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 403/814] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.076816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.077394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.077672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.086750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.087651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.088100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.088546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.090154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.090620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.092324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.097484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:17.099413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:17.102408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/815] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:25:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.103383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.144887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.196058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 404/816] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:17.196058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.224496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.243981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4194c-cbd2-4fb2-845a-29662e462b83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.279355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.280456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.280880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.281328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.289110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.290952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:17.305509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:17.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:17.307682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:17.309774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/817] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:17.310765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da40dc-8d4b-4e6b-8b7b-51ee1a82d97c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.310898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.347663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.348793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.349210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.349616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.360928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.362608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:17.381390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:17.382503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:17.383490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:17.383490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:17.383490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:17.383490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:17.389751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 405/818] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:17.390971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.434524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0430652-e771-492d-80cd-0f0a865dda64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.443808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.483312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.484309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.484592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.484791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.485436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.486398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.488491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:17.505024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:17.506091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:17.507067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/819] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/services/network => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:17.508142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.513754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a970-97b2-45a6-82a3-56d6cf912a3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.555945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.556937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.557436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.557873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.558820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.559681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.561673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.576577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 406/820] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.577627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.639375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d3cbc-ce8c-4b64-bcc4-951fc8c9a595 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.687580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.688510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.688782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.689149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.689994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.690640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.692422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.702851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52371aba-be3c-4bd9-ba81-f24479cbe415 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.709509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/821] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.710545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.746786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.747465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.747743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.748017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.748755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.749057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.750283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.751065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.757720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 407/822] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.758603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.761400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.831714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb799ea1-b258-43b4-a735-f74156738fb4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.866435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.867463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.867935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.868433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.869458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:17.869585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08af15-c426-4d59-80f8-155d22b8841b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.870414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.872128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.878923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/823] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:17.880106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.944195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.945407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.945913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.946379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.969266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.971182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:17.999373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:18.000406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:18.001984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 408/824] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:18.006511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.036340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:18.053933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103e40f-76c8-418f-be16-0ad1781f6fc5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.090449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.091513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.092123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.092620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.093531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:18.094132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.095865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.102097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/825] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:18.111139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.128414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:18.189093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa69813-c571-48fc-9165-bb96169bcee2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.231144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.231860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.232185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.232565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.233449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:18.233760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.235192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.253555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a779-6794-4aec-8d04-7cd8a731490f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.257093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 409/826] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:18.262188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.299361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c69-7165-49a8-bce4-436302537700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.301217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c69-7165-49a8-bce4-436302537700 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.301654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c69-7165-49a8-bce4-436302537700 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.302100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80763c69-7165-49a8-bce4-436302537700 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.303005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763c69-7165-49a8-bce4-436302537700 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.304583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80763c69-7165-49a8-bce4-436302537700 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.331639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/827] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:18.380695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710945b-58ea-4b1f-9a0f-bbd958e95e10 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.405625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:18.438251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.439479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.440133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.440557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.453135 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.454960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:18.475121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:18.476309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:18.477407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:18.477407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:18.477407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:18.477407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:18.479343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 410/828] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:18.484972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c69-7165-49a8-bce4-436302537700 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.489299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:18.595325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80763c69-7165-49a8-bce4-436302537700 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.630943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.631621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.631901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.632206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.632930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:18.633237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.634622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.647440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/829] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:18.654009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.765043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699cebb-b3a4-417a-a05b-28784ff1b0b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.803932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.804767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.805050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.805325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.812599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.814128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d049a774-0397-48f4-8dbc-4af5f803f5df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:18.829159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:18.830134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:18.831260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 411/830] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:18.836630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:18.906912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:18.956702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d552027-d5bf-4cfa-9d82-e84bdcfc3d3f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:18.960704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7fd-0b6a-4115-9fc2-c2d2c7666c1a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.961181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7fd-0b6a-4115-9fc2-c2d2c7666c1a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.961271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beda7fd-0b6a-4115-9fc2-c2d2c7666c1a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.962876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 412/831] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:18.968088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a8265-d868-498d-82c1-bf637adae413 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.968385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a8265-d868-498d-82c1-bf637adae413 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.968664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a8265-d868-498d-82c1-bf637adae413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.986741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-394a8265-d868-498d-82c1-bf637adae413 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:18.988023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-394a8265-d868-498d-82c1-bf637adae413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:18.994862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:18.995529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:18.995805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:18.996110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:18.996890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:18.997196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:18.998642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.013430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/832] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:19.014345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.021935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.066428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 413/833] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:19.091465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a8265-d868-498d-82c1-bf637adae413 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.129449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a11-95d5-4d57-90fd-19186c90e6fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.154606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.166619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.167616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.167894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.168211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/69a0a21790ed496ba0e1e397a9daea09/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.184855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=69a0a21790ed496ba0e1e397a9daea09, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.186317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fc2c1-e262-42bd-a1f5-b35c791b880a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.202517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/834] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:19 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/69a0a21790ed496ba0e1e397a9daea09/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:25:19.214327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a8265-d868-498d-82c1-bf637adae413 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.256618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.257586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.257865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.258239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.271602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:19.279760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:19.280786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:19.281637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:19.282263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4a24-69c5-4fae-9deb-7aaf71cc3eb9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.282263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4a24-69c5-4fae-9deb-7aaf71cc3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.282391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 414/835] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:19.282458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e4a24-69c5-4fae-9deb-7aaf71cc3eb9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.285764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/836] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:19.286838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.288411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.288697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.288971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.306453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:19.307668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:19.312022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.384183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/837] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:19.399305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.421387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b81dec-bcf4-41dd-85a0-5e8166bed0e5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.459348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.460023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.460343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.460620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.461366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:19.462130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.463547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.472844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 415/838] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:19.478313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.522957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86722bcc-1467-4bb9-9cfa-793a5092d2ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.525798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.570904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.571915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.572232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.572499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.573136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.574081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.575856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:19.593309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:19.594365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:19.595299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:19.596089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/839] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:19.602690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.605357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e26100-7594-48e8-9d25-66b6dbefd4b5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.650617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.651819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.652268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.652660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.669492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.671621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:19.696797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:19.697714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:19.698619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:19.698619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:19.698619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:19.698619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:19.700449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 416/840] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:19.705181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.727682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52580e-f06e-4336-a6d5-a789b0757eb4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.765794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.766469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.766756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.767146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.768045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:19.768366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.769927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.779348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/841] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:19.781334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.816503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.840938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6e239-0aa6-469e-8ac5-11e34049b06a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.878720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:19.884908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.886025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.886835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.887257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.888401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:19.888846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.891091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.896671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae98bf5-ccf3-40dd-a4b7-f261801d06b1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:19.906309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 417/842] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:19.912000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:19.938867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:19.940225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:19.940225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:19.940421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:19.941241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:19.941612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:19.943173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:19.956801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/843] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:20.028278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8828f-5da3-45bd-875a-2f3e7e7b7a93 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:20.067782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:20.068930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.069408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.069858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.078352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:20.080217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:20.096475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:20.097471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:25:20.098406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:20.100940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 418/844] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:20.109407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:20.230601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:20.236665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223592d-f78d-4faa-b12f-aab58d0292f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:20.271326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:20.272312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.272731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.273139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.274108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:20.274552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:20.276173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccab0f-e9e8-448b-a148-597cc68e557d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:20.294915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/845] 10.222.0.55 () {56 vars in 1135 bytes} [Tue May 24 15:25:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:20.300695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:20.357144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:20.413693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164824c-c1eb-4856-a12f-ac678bbae437 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:20.448334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:20.449236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.449654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.450080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.527472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:20.529239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:20.602531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:20.654923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 419/846] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:20.660557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:20.661181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d8b1-58d9-4b0e-acbd-c3c26f54e7e5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.661471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d8b1-58d9-4b0e-acbd-c3c26f54e7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.661744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734d8b1-58d9-4b0e-acbd-c3c26f54e7e5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.670414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/847] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:20.674534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.675066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.675561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.694641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:20.695842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:20.779956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/848] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:20.789566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf0a6e-4a8a-4f10-90cc-fd54b9c19cc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:20.801673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:20.827555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:20.828891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.829423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.830217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.844105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:20.846381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:20.869038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:20.870003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:20.870927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:20.870927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:20.870927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:20.870927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:20.873315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 420/849] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:20.878347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:20.884104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:20.905938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:20.928775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3059056-5ba8-4551-850d-3bae41bf64cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:20.974499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:20.975387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:20.975675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:20.975995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:20.976696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:20.978626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:20.995409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:20.996427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:20.997280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:21.000177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/850] 10.222.0.55 () {56 vars in 1132 bytes} [Tue May 24 15:25:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:21.003261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.013073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c4bd2-1994-474d-84c9-822a6264031e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.054683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.055329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.055592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.055852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.057009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.057326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.058700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.077926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 421/851] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.079742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.137251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c0f36-5b26-437a-843d-6927aeb8ad4d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.162252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:21.180980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.182071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.182071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.182285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.182964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.183268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.184983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.194817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/852] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.196206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.216617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:21.218007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7031a-9933-4f95-923d-ba7b6d9817b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.253837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.254638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.254918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.255188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.262608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.264140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:21.281393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:21.282378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:21.283345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:21.287702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 422/853] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:21.288788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.341380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d976-7322-4c4e-82eb-60dad57423d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.388560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.389866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.390591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.391144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.392463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.393250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.395437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.397958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5639bac-90e7-421a-b618-aaa9c890c291 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.409083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/854] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.429359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:21.441762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.442448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.442729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.443002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.443802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.444147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.445626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b59880-efc7-41a7-b0c6-cc11bc951d5a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.459972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 423/855] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.466120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.468406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9af24-2188-4646-b65f-f0c4bf8e3986 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.468724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9af24-2188-4646-b65f-f0c4bf8e3986 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.469012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9af24-2188-4646-b65f-f0c4bf8e3986 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.470634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 424/856] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:21.476437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.477069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.477548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.495969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:21.497566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:21.568228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:21.584471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 425/857] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:21.594496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a88e3-9894-4de2-a019-4696bc18e448 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.602593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.644538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.644538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.644538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.644896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.645674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=eeb122c4f54840a98cd9fc932fb2856c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.645985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.648021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.656003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/858] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/role_assignments?user.id=eeb122c4f54840a98cd9fc932fb2856c&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.732286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1c098-bfea-442d-8d63-3149168648ce None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.748586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:21.773001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.773820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.774098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.774365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.786795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:21.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:21.796149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:21.797005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 426/859] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:21.804460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:21.914479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd894bc-9bcb-4fe1-a364-9a02cd485ac1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:21.949783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:21.950716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:21.951135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:21.951542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:21.952447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:21.952885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:21.954459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:21.963515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/860] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:21.969186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.077697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ede10-a606-441e-9c64-2eb8bf5cd70c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.118443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.119677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.120204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.120675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.132714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.134505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-67f328e9-0bc2-48b6-86a0-e706dedbe7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:22.152515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:22.153457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:22.154360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:22.154360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:22.154360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:22.154360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:22.156635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 427/861] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:22.168076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.173123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:22.272084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a70ec-4686-427f-8cee-a3a10430f0ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.290279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a2aa-441b-469d-84da-08fed331afcb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.290608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a2aa-441b-469d-84da-08fed331afcb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.290835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64a2aa-441b-469d-84da-08fed331afcb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.292522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 428/862] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:22.298744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.299173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.299594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.314416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.315309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.315842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.316297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.317150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:22.317399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:22.317772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.318796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:22.319482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.332857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/863] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:22.344048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.345364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:22.405061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:22.416184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 429/864] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:22.442974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.450381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe26614-0542-441c-bab0-261d6f150151 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.485971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.486797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.487061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.487330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.494864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.496344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:22.511776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:22.512855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:22.514531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/865] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:22.518947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.556974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c9958-3d52-40e2-8c29-aca5998a72bf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.596412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.597095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.597377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.597651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.598382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.599772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.620190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 430/866] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:22 2022] POST /identity/v3/services => generated 251 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:25:22.632261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd98930-84b8-41e7-8ff7-1a3c25dbc1b9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.658127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:22.680614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.681295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.681617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.681938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.682844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:22.683485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.684623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.710863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/867] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:22.716394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:22.738321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:22.834121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a99559-a1f5-4675-b6bf-790a6cfce19a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:22.871356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:22.872109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:22.872386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:22.872656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:22.944574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:22.946265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:22.964079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:23.054736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 431/868] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:22 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:23.060193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.166207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab9998-e9c8-4f09-82d6-c24ddf9a5f16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:23.209313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:23.210115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.210386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.210653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.221621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:23.223084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:23.241519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:23.242518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:23.243976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:23.243976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:23.243976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:23.243976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:23.243976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/869] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:23.248309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.354403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618978d6-8de6-4e2c-b85b-a244fe88b83f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:23.371433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:23.389988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:23.390676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.390930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.391295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.392326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:23.392657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:23.394036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:23.406501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 432/870] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:23.412084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.442070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:23.524487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa139c4-ec11-4857-9e0c-abcab639e557 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:23.568611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:23.569419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.569696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.569969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.577052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:23.578513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-094df7dc-3d81-41eb-945d-7077a0577264 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:23.592709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:23.594774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:23.596472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/871] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:23.599636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.699247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec2ce-8ad8-479d-b6ee-d65f3cc56178 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:23.704891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf703fd-b1d0-45d1-a9f6-be5a319748d7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.705156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf703fd-b1d0-45d1-a9f6-be5a319748d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.705421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf703fd-b1d0-45d1-a9f6-be5a319748d7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.707024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/872] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:23.712390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.712737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.713505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.735324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:23.737063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:23.740292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:23.741888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.742235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.742572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.743424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:23.743796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:23.745372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:23.762164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 433/873] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:23.768467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.814476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:23.832317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/874] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:23 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:23.859689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:23.880891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:23.909089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46139ac-65c9-4b7f-b8d9-9e69b534f949 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:23.951557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:23.952509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:23.952859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:23.953219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:23.954264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=69a0a21790ed496ba0e1e397a9daea09) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:23.954975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:23.956832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb0ada-f7e2-4559-9ca1-c75c95cee974 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:23.964724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 434/875] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:23 2022] GET /identity/v3/role_assignments?user.id=69a0a21790ed496ba0e1e397a9daea09&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:23.975563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296b5c6-07c9-416e-b5eb-782cba6bd141 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:24.016402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:24.017824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.018276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.018714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.019599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:24.021198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345133b1-bf87-4af4-b3b7-14cc7132e473 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:24.041270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/876] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:23 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:25:24.048287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fa4d-b612-429d-a9d3-9e41e1feb440 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.048287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fa4d-b612-429d-a9d3-9e41e1feb440 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.048287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fa4d-b612-429d-a9d3-9e41e1feb440 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.048287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 435/877] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:24.052732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.053300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.053747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.074751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:24.076988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:24.169576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/878] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:24.191798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257fa4d-b612-429d-a9d3-9e41e1feb440 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:24.309144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:24.314853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257fa4d-b612-429d-a9d3-9e41e1feb440 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:24.350787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:24.351857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.352300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.352739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.353535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:24.355036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:24.367584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:24.368705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:24.369544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:24.372106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 436/879] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:24.375980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:24.387778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:24.496986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a9069-6ab4-4317-9393-d572df328017 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:24.532736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:24.533452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.533728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.533994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.534709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:24.535006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:24.536417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e37d57-2444-476c-8d2e-64ebbfe320a6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:24.541824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/880] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:24.547754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:24.655122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d425bf-24af-43e7-bca8-338779d081ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:24.691101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:24.691787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.692088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.692370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.693086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:24.693396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:24.694788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43ae1c-34ba-448e-bcb4-1c98183f5d64 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:24.704178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 437/881] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:24.978758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4ccc-76c0-452a-8e06-7813d15febae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.979169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4ccc-76c0-452a-8e06-7813d15febae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.979351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df4ccc-76c0-452a-8e06-7813d15febae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:24.980945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/882] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:24.986410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:24.986699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:24.986980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.003985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:25.005212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:25.028621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:25.088164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 438/883] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:25.110516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df4ccc-76c0-452a-8e06-7813d15febae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:25.179914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:25.218874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df4ccc-76c0-452a-8e06-7813d15febae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:25.255657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:25.256407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:25.256682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:25.256950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.257603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:25.259058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:25.271800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:25.272951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:25.273857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/884] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:25.279102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:25.383079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662bd37-af30-44a4-9484-8bb4b984a3e3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:25.419092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:25.419765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:25.420058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:25.420341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.421055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:25.421363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:25.422741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:25.427975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 439/885] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:25.433615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:25.532243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2899-b8fd-45bb-8eb5-7f777c1c6856 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:25.566361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:25.567046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:25.567329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:25.567605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.568360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:25.568670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:25.570130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:25.572351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:25.575559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/886] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:25.581052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:25.607142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:25.695956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7a8d-44a1-434b-8753-c659d95bcd12 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:25.730792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:25.731515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:25.731799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:25.732119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.732944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] RBAC: Adding query filter params (interface=public, service_id=8bab2bb72b21467688a688c48a1445a4, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:25.733256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:25.734634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe054161-1ab8-42d4-baf9-b8a810151a27 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:25.742662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 440/887] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:25 2022] GET /identity/v3/endpoints?service_id=8bab2bb72b21467688a688c48a1445a4&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:25.748361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:25.852466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ee5e1-c8f6-494b-88c7-defbe18c2a35 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:25.890545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:25.891317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:25.891647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:25.892314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:25.893221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:25.894833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902a1b2c-0cad-4b24-ac42-73615a63b679 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:25.901499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/888] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:25 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:25:26.371488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bde63-8684-46f1-810e-2537c8267626 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.371488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bde63-8684-46f1-810e-2537c8267626 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.371488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bde63-8684-46f1-810e-2537c8267626 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.371488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 441/889] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:26.379270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b3650-b607-4ef3-9201-272cd97b3aa8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.379270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b3650-b607-4ef3-9201-272cd97b3aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.379270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b3650-b607-4ef3-9201-272cd97b3aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.395975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ce3b3650-b607-4ef3-9201-272cd97b3aa8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:26.397514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ce3b3650-b607-4ef3-9201-272cd97b3aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:26.454474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:26.466186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/890] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:26.477472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ebe6-e5aa-424c-90ec-3e49ba36e02a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.478002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ebe6-e5aa-424c-90ec-3e49ba36e02a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.478002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722ebe6-e5aa-424c-90ec-3e49ba36e02a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.479536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 442/891] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:26.484940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.485475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.486120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.494011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722ebe6-e5aa-424c-90ec-3e49ba36e02a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:26.504388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:26.505576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:26.584153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/892] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:26.595523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:26.602103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:26.619712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722ebe6-e5aa-424c-90ec-3e49ba36e02a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:26.670250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:26.671038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.671321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.671597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.685676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:26.692004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:26.692982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:26.693875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 443/893] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:26.698951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:26.733180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aba38-9f9e-4b86-8f77-8a8f558e9d12 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:26.762142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:26.783484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:26.784375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.784657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.784930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.785602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:26.787360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:26.806150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:26.807217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:26.808110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:26.812172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/894] 10.222.0.55 () {56 vars in 1132 bytes} [Tue May 24 15:25:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:26.815128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:26.837949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c34fd-e5ee-4060-be89-28d3e0e7cb40 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:26.880446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:26.882033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.882791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.883266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.884881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:26.885414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:26.887511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:26.899026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 444/895] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:26.908850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:26.923110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:26.940296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb38766-7ceb-4fe3-a7ba-626b9e621d47 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:26.983113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:26.984202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.984680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.985168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.986281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:26.986766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:26.988336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18f7ec6-0f19-48fb-b75d-19ec89d2d004 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:26.993648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/896] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:26.995084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef73c-4ac5-474d-a636-70e7f7672759 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:26.995357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef73c-4ac5-474d-a636-70e7f7672759 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:26.995629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ef73c-4ac5-474d-a636-70e7f7672759 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:26.996986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/897] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:27.000967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ef73c-4ac5-474d-a636-70e7f7672759 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.029275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c511b9-c6a7-420a-866f-c71a68aae63b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.056211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.068802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.069599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.069871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.070140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.081334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.082806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fee12adb-954f-42b9-8385-069adcaf1984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:27.100860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:27.101931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:27.102855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:27.102855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:27.102855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:27.102855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:27.102855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 445/898] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:27.103446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.103727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.104001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.117481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ef73c-4ac5-474d-a636-70e7f7672759 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.124080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:27.125775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:27.151785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.152580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.152889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.153164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.153867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:27.154166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.155535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.160883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/899] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:27.161500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.193407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.208673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 446/900] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:27.209932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.267112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225b7b2-a884-4c28-a17b-f7ec00443af7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.284157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.310374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.311145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.311483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.311823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.312646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:27.312955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.314385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.328078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/901] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:27.328078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.335670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811cbe2-a657-40bc-89ef-02621665ccf0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.370179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.370850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.371109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.371381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.372197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] RBAC: Adding query filter params (interface=public, service_id=c69d448b0c2a4bbe96f047f4c560b46c, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:27.372665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.373927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.383317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 447/902] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/endpoints?service_id=c69d448b0c2a4bbe96f047f4c560b46c&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:27.383317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.434456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba296dd-be61-497c-90a7-4d44fd4168d4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.449498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.474243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.475343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.475587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.475790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.476540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.478074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.494551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/903] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:27 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:27.495173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.510351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73288c-59c9-46c0-9c53-84157873d2a1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.534943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.574282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.575111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.575383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.576496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.588886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.594417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:27.617996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:27.619030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:27.620023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:27.626456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 448/904] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:27.628307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.631248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec5a50-82d9-4c29-8c8f-c83b6df2e628 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.677505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.678177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.678452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.678718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.679567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.680948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.696056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/905] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:25:27.757755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480f84-04f3-4451-915b-1ee7e37c0b8a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.776576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.797060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.797752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.798110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.798438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:27.799437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:27.799750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:27.801274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d2a6-51c4-478d-b321-c7c751438596 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:27.822211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 449/906] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:27.828036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:27.932537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:27.939039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1b840-e8d3-42fd-87ee-ccc5bea1434a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:27.977754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:27.978472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:27.978762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:27.979036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.055747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.057157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.153003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.158486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4205a-6e59-4d35-b10f-ee852d00b822 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.158770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4205a-6e59-4d35-b10f-ee852d00b822 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.159033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4205a-6e59-4d35-b10f-ee852d00b822 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.160947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 450/907] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:28.164192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/908] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:28.166327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.166610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.166879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.169868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.186025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:28.187855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:28.268420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 451/909] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:28.285166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12108bf6-d892-435a-82ef-7deada4cee47 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.290418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.323008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.323746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.324018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.324319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.338371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.340571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:28.365664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:28.366809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:28.367900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:28.367900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:28.367900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:28.367900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:28.372022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/910] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:28.373421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.396929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.400824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26472ab-7459-40bb-aac5-1c1fb0151071 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.437843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.438852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.439218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.439581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.440361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.441828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.443576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:28.462768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:28.464707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:28.466099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 452/911] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:28.473014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.507731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7603c7e-6a74-451e-a372-8d19df7003bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.554980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.555656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.556335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.556629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.557631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:28.557941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.559732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.578431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/912] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:28.579458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.587373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be603-5e8c-438e-9c08-1d5e34446c49 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.633956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.634961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.635309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.636068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.637127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:28.637501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.639859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.645979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 453/913] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:28.652380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.694196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.701320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae6a1-b62d-43ec-920b-03668b74682f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.716319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.750796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.752001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.752311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.752587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.761112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.762994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.778664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2b9d-2122-438d-b12c-745bc9bf1ba3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:28.784865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:28.785801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:28.787034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/914] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:28.792990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.822999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.824015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.824537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.825100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.826141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:28.826645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.828296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.835202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 454/915] 10.222.0.55 () {56 vars in 1114 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:28.843248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.900043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f040ca-0f1f-4a9a-b109-1796345ff008 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.936388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:28.937795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.938354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.938738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.938977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.939943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:28.940412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.942708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.947954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbc258-6ef5-417a-b476-ff1c327b5430 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:28.955682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/916] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:28.963891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:28.982629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:28.983424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:28.983708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:28.983976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:28.984800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:28.986233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cc22b-2090-47ed-b682-88d11ac8d8e2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:28.994182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:29.016336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 455/917] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:29.080358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8211900-5422-4db0-8241-a18e624526ed None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:29.123891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:29.124529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:29.124809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:29.125086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:29.125916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] RBAC: Adding query filter params (role.id=0c94a10d75ed4085b1f6d83c6c645e15, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=69a0a21790ed496ba0e1e397a9daea09) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:29.126257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:29.127720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a83fee-634c-424f-afe5-80a5c33d00e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:29.134416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/918] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:28 2022] GET /identity/v3/role_assignments?user.id=69a0a21790ed496ba0e1e397a9daea09&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=0c94a10d75ed4085b1f6d83c6c645e15 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:29.898624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645788-a5db-4c34-b7da-b377a081422c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:29.899177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645788-a5db-4c34-b7da-b377a081422c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:29.899177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25645788-a5db-4c34-b7da-b377a081422c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:29.900899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 456/919] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:29.906463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:29.906914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:29.906991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:29.924846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:29.926029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:29.952138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39adb2-cd5c-4dc9-9e7c-c41d73906a08 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:29.952447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39adb2-cd5c-4dc9-9e7c-c41d73906a08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:29.952708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39adb2-cd5c-4dc9-9e7c-c41d73906a08 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:29.954446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 457/920] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:29.959860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:29.960169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:29.960452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:29.978942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:29.980626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:29.997632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/921] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:30.019875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.039976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.064119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 458/922] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:30.088609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.128881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253ee71-5ecf-422c-978f-2259ce27ed5b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.165739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.166552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.166831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.167103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.167875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.169709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.176725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:30.186233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:30.187301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:30.188219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/923] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:30.194393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.210998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43f1e7-5398-4f9e-bdc0-9c2dad4b033a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.257234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.258133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.258412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.258683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.259746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.260982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:30.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:30.276613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:25:30.277621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:30.288080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 459/924] 10.222.0.55 () {56 vars in 1132 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:30.288080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.294863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.315851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d814d-7904-4672-8345-6060677ab0ac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.351045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.351719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.351994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.352308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.353253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:30.353554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.355490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.366618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/925] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:30.368326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.399902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850efbf-03af-4190-a869-3ae26f0444cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.444464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.445178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.445457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.445983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.446692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:30.447799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.449212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.455896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 460/926] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:30.463394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.468815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.483837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f084feb-6e87-417d-91a8-aa461db85c60 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.519038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.519739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.520087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.520367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.521076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:30.521381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.522772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.532154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/927] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:30.534250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.548719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.603428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8082dab-7268-4b83-9459-cf0eeb525220 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.642501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.643002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.643292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.643566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.644325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:30.644632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.646254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.655900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 461/928] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:30.657438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.667550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553682aa-dc66-4abc-9640-c478a30c33b4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.709245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.710366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.710852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.711326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.712375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] RBAC: Adding query filter params (interface=public, service_id=800fb5f2f8cf4d3381e10f57d1bcbd05, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:30.713122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.714872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.721930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/929] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/endpoints?service_id=800fb5f2f8cf4d3381e10f57d1bcbd05&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:30.728635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:30.732109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.783237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb465d3-16f3-4c32-8580-4f60737f1683 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.816940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:30.835984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.837282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.838155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.838936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.840165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.842416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba44c87-d14e-4018-8e6e-6df445126359 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.843485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5e380-6fbc-45ba-ba58-4417787d79e9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:30.873554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 462/930] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:30.898341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:30.899436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:30.899984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:30.900498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:30.901600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:30.903348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad730b-7cb0-46ca-910d-0ecbd81d83be None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:30.912374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/931] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:30 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:25:31.345980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dca31-111b-4659-86ef-a373e60d6e23 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.346512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dca31-111b-4659-86ef-a373e60d6e23 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.346512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dca31-111b-4659-86ef-a373e60d6e23 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.348073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 463/932] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:31.353489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.354011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.354483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.361531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036134c7-100b-461a-9542-f4f51e3409da None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.361814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036134c7-100b-461a-9542-f4f51e3409da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.362091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036134c7-100b-461a-9542-f4f51e3409da None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.363709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 464/933] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:31.368766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.369050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.369331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.373765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:31.375172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:31.387072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:31.388558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:31.459857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 465/934] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:31.472046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/935] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:31.481827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:31.489121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:31.544421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:31.565062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:31.596955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b5db5d-abdb-4410-825e-ad6db2a9b3a0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:31.622551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b84071-9eba-4b47-91fe-2daf3e014e69 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:31.639519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:31.641369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.642721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.643205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.658521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:31.662133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:31.665979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.665979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.665979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.665979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:31.668116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:31.669860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING keystone.server.flask.common [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:31.671528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:31.672478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:31.673347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:31.681698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 466/936] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:31.682306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:31.704192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:25:31.704192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 15:25:31.704192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [parameters: {'id': 'f955b5a48b734f59ab9ef5e1c19e8624', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 15:25:31.704192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f955b5a48b734f59ab9ef5e1c19e8624.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f955b5a48b734f59ab9ef5e1c19e8624. May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:25:31.727144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 24 15:25:31.728224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return project_ref.to_dict() May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() May 24 15:25:31.729396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:25:31.730506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:25:31.731613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:25:31.732651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': 'f955b5a48b734f59ab9ef5e1c19e8624', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.733734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:31.734716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict( May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f955b5a48b734f59ab9ef5e1c19e8624. May 24 15:25:31.736006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.739713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/937] 10.222.0.55 () {60 vars in 1140 bytes} [Tue May 24 15:25:31 2022] POST /identity/v3/projects => generated 262 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:31.745357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:31.818612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889fa71-9073-4eda-bed8-d94a7d51dbfc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:31.836816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:31.858551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e9bcc-39f9-4442-977c-aa0ee3656c5b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:31.863827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:31.864738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.865260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.865729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.867100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:31.867581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:31.869555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:31.884710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:31.884874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 467/938] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:31.887602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:31.902220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:31.902877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:31.903158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:31.903429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:31.910838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:31.912626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:31.927700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:31.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:31.929536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:31.934157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/939] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:31.934878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.025595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5758f75-b983-49b9-af24-315123b50a07 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.064635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.065453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.065729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.066040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.068565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387e480-44ce-4d74-9273-9515b5362f1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.077306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.078757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:32.096534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:32.097521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:32.098407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:32.098407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:32.098407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:32.098407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:32.102750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 468/940] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:32.103733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.119696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.120975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.121957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.122438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.123603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:32.124362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.126427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.138250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:32.156144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/941] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:32.224863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91af8d-2b7c-4959-aa07-0291582ecb10 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.262702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.263723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.264236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.264709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.265621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:32.266151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.267956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.281439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 469/942] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:32.290739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.340399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:32.402464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8fe3-07f5-48a4-8be2-08fe9668e822 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.436718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.437537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.437810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.438072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.446178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.447628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:32.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:32.463498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:32.464506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/943] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:32.469569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.559501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:32.572754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857724d-ffbd-4b4a-9149-ee62d943b3d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.608339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.609394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.609866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.610439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.611591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:32.612117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.613775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.632401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 470/944] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:32.640077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.744245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:32.747596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236fe3d-ec37-40aa-ad55-0c75a277a128 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.786074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.786877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.787158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.787430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.801632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.803130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:32.821519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:32.822472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:32.823378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:32.823378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:32.823378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:32.823378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:32.823378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/945] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:32.828728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:32.933153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb72b1d-10a2-449b-af2e-70e85ecf050e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:32.976067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:32.977179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:32.977678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:32.978145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:32.979089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:32.979600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:32.981271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff1681-1359-4046-a195-aca115a0c9e2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:32.996191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 471/946] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:33.002370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.118842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b515e5a-e72c-4574-832e-792b0474f04f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.155599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.156441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.156723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.157018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.165000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.166474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.168283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bbfc6-e77b-4003-9c5c-34bd18851939 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.169134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bbfc6-e77b-4003-9c5c-34bd18851939 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.169597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bbfc6-e77b-4003-9c5c-34bd18851939 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.171412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 472/947] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:33.176637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631aa3-d281-417c-afc2-01c7f6e2aa82 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.176926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631aa3-d281-417c-afc2-01c7f6e2aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.177205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2631aa3-d281-417c-afc2-01c7f6e2aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:33.182436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:33.183351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.184699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/948] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:33.189862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.194000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.199507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f2631aa3-d281-417c-afc2-01c7f6e2aa82 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:33.201006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2631aa3-d281-417c-afc2-01c7f6e2aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:33.206115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.279961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 473/949] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:33.281280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dc55-9e18-4193-a537-3a906bb0bf8d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.281568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dc55-9e18-4193-a537-3a906bb0bf8d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.281958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dc55-9e18-4193-a537-3a906bb0bf8d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.283887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 474/950] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:33.289018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.289302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.289574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.302457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5cf4c-4d1c-410c-be07-e4cfc84e2f91 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.311159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:33.312476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:33.337845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.338506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.338780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.339051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.339821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:33.340152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.341592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.353025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/951] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:33.353828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.385335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 475/952] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:33.386430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.430937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.461130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.462155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f651749-4b74-4473-a749-fb4a202bb7ce None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.498801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.499048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6048-e289-418f-83c0-a80b39139f91 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.499635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.499895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.500205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.500914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.502903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:33.515121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:33.516403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.517562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/953] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:33.519177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.545127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.545775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.546051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.546333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/69a0a21790ed496ba0e1e397a9daea09/roles/0c94a10d75ed4085b1f6d83c6c645e15` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.549590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=69a0a21790ed496ba0e1e397a9daea09, role_id=0c94a10d75ed4085b1f6d83c6c645e15)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.550997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.576202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 476/954] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:33 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/69a0a21790ed496ba0e1e397a9daea09/roles/0c94a10d75ed4085b1f6d83c6c645e15 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:25:33.576202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.639632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c228867-3328-4bcd-a5be-12a252938579 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.690118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.691230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.691623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.691899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.692602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.694016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.698108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:33.712198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:33.713422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:33.714289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/955] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:33.720003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.736232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da331d8-6eac-49e8-b5a9-7d2a022a7f8a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.777079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:33.793022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.794039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.794299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.794567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.795323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:33.795566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.797044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.802115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 477/956] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:33.807842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.855561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecf4ef-d90b-41bb-a607-e6d811a409ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.892143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.893194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.893413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.893791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.894569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:33.895231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.896734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.902626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/957] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:33.909633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.924427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60ac33-f031-4ddb-adc7-1e9519364644 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:33.962201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:33.962894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:33.963222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:33.963496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:33.964267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:33.964574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:33.966154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:33.971945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 478/958] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:33.977584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:33.992260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:34.038767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f110fa-03e6-4659-97e0-fc55a22d1595 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:34.041108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:34.081587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:34.082901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.083345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.084057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:34.084905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:34.085494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:34.087663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bed0dc-3098-441f-8014-c4eff48cbe81 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:34.092817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64545e-dba3-4c08-aed8-cf32d3ca2e15 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:34.100510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/959] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:33 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:34.140929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:34.141654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.141923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.142177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:34.143049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:34.144493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024cd7e-6681-4cc6-9071-5e438c44efad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:34.167730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 479/960] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:34.547796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659d05-700a-42cc-bf45-f590b9b8d1d0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.548233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659d05-700a-42cc-bf45-f590b9b8d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.548409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b659d05-700a-42cc-bf45-f590b9b8d1d0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:34.550071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/961] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:34.555636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.556113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.556209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:34.573436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:34.574628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:34.651134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 480/962] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:34.676640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659d05-700a-42cc-bf45-f590b9b8d1d0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:34.787070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b659d05-700a-42cc-bf45-f590b9b8d1d0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:34.815047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:34.822977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:34.823654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.823931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.824240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:34.824945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:34.826383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:34.841620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:25:34.841620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 15:25:34.841620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [parameters: {'user_id': '02c01f496092447a91b4a99c7a1de330', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 15:25:34.841620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 24 15:25:34.849386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:25:34.850264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:25:34.851311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:25:34.852231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = connection._execute_20( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:25:34.853153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:25:34.853995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'user_id': '02c01f496092447a91b4a99c7a1de330', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:34.854857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 24 15:25:34.855666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:34.856756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/963] 10.222.0.55 () {60 vars in 1131 bytes} [Tue May 24 15:25:34 2022] POST /identity/v3/users => generated 160 bytes in 175 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:34.860011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:34.894667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:34.962890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d34e9-284d-4f2f-ad38-d9b17886a7c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:34.997983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:34.998852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:34.999136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:34.999411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:35.006953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:35.008433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8d464213-021b-4314-8c07-e0e5bc00c6b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:35.023144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:35.024153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:25:35.025125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:35.028166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 481/964] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:35.030878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:35.133196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cb00a-e519-4260-84d2-c3a405d8174c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:35.166837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:35.167506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:35.167786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:35.168075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:35.169190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:35.169488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:35.171123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64815b6-bdff-4445-bf2c-058dbc89c78e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:35.209094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/965] 10.222.0.55 () {56 vars in 1101 bytes} [Tue May 24 15:25:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:35.835017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7636-a461-4afd-bd0e-a52ae29d8614 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:35.835542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7636-a461-4afd-bd0e-a52ae29d8614 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:35.835542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ba7636-a461-4afd-bd0e-a52ae29d8614 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:35.836999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 482/966] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:35.842460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:35.842711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:35.843020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:35.860714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:35.862226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:35.906689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:35.931953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/967] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:35.955534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba7636-a461-4afd-bd0e-a52ae29d8614 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.060541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.068838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ba7636-a461-4afd-bd0e-a52ae29d8614 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.104151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.104975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.105257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.105525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.116230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:36.124379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:36.125316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:36.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 483/968] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:36.133292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.242093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddba19-7ec3-4db8-b088-27f4c1ffc8d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.290991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.291690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.291971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.292230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.293009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:36.293610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.295054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b454fe-3319-49bd-9830-f3fa6b0cb107 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.304487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/969] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:36.309911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.384660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097edee-3beb-4abf-be7f-5c012bf0d1b3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.385008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097edee-3beb-4abf-be7f-5c012bf0d1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.385200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097edee-3beb-4abf-be7f-5c012bf0d1b3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.386820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/970] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:36.391863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db09b6a-3b02-40c7-9594-c3dfef174120 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.392151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db09b6a-3b02-40c7-9594-c3dfef174120 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.392466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db09b6a-3b02-40c7-9594-c3dfef174120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.408940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3db09b6a-3b02-40c7-9594-c3dfef174120 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:36.410123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3db09b6a-3b02-40c7-9594-c3dfef174120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:36.414315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507fc51-717d-41f5-9a8e-27e96db1c920 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.447140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.451811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.452899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.453648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.454000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.465799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.467240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.479432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/971] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:36.480874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761962-a092-4166-b373-f010b8cca617 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.481161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761962-a092-4166-b373-f010b8cca617 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.481438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55761962-a092-4166-b373-f010b8cca617 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.482848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/972] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:36.484378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:36.485351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:36.486823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:36.486823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:36.486823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:36.486823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:36.486823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 484/973] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:36.489299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.489580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.489848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.492055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.504124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.507206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:36.508564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:36.574828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/974] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:36.575621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.601736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8a8bf-313b-4106-a375-857db7e35ec0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.636320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.636988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.637269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.637544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.638266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:36.638573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.639957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.651661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 485/975] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:36.652477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.673004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.676172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4f20d-4449-4175-8fb8-dd612cc6de46 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.711858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.712555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.712818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.713083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.713794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.715187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.720586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.729825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/976] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:36 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:36.730629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.765974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30009e61-8cf8-4c46-a77f-ca81278b1fd6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.812376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.813206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.813904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.814229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.814837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.816263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:36.830323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:36.831431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:36.832533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 486/977] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:36.837745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.851447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faf2f5-a472-4ae9-851b-e201e1db279a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.892722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.893928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.894371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.894798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.903638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.905413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:36.921681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:36.922809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:36.923937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/978] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:36.928990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:36.955755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.957225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a46f868-1da7-4330-81a2-046da60f005c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:36.985993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:36.994620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:36.995281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:36.995583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:36.995923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:36.996842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:36.997144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:36.998510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.004346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 487/979] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:37.009396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.044919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da0dde-bb05-4c36-88a4-0b5ed9218300 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.081518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.081850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.082137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.082409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.083353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:37.083661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.085164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.103961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/980] 10.222.0.55 () {56 vars in 1149 bytes} [Tue May 24 15:25:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:37.109426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.116508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa0b92-e191-4a22-b95b-80f7a3ca9cca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.152586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.153254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.153536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.153811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.154519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:37.154829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.156224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.161368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 488/981] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:37.167105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.195201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:37.220473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0685803-5523-4d44-9053-7be25c0cdb75 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.233515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:37.258016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.259054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.259334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.259603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.283202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834236f-3598-4f85-8fbf-778d4fb4b5fc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.323122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.323703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.323983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.324286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.325519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] RBAC: Adding query filter params (interface=admin, service_id=800fb5f2f8cf4d3381e10f57d1bcbd05, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:37.325838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.327217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.333048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 489/982] 10.222.0.55 () {56 vars in 1249 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/endpoints?service_id=800fb5f2f8cf4d3381e10f57d1bcbd05&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:37.338356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.340949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.342423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.424662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:37.447184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1326e957-0ad8-46fd-8139-4bbb71ce1199 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.449139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/983] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:37.454868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.467667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:37.483156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.483811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.484143 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.484415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.485138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.486506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.493348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 490/984] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:25:37.575640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e340576-1f28-45ea-aab6-b1b5fc0a012a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.614299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.615125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.615388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.615789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.627189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.628769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:37.648261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:37.649268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:37.650197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:37.650197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:37.650197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:37.650197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:37.650197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/985] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:37.655324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.770056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7262682-5a7b-4710-8e97-bd0a5e9a07a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.805071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.805738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.806015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.806278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.807055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:37.807353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.808813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.825341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 491/986] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:37.826927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:37.859636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:37.935554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5837f40-020c-4f36-8c76-8300bc1b4d7d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:37.970537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:37.971338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:37.971631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:37.971895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:37.978848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:37.980326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:37.996222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:37.997167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:37.998094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/987] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:38.001608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:38.050711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:38.106557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a63804-4224-43d7-8f90-2efd369f9cd0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:38.141186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:38.142043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:38.142298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:38.142599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:38.143376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:38.143680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:38.145165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e74f4-b0cc-4ed2-91d2-2b937ae0fe0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:38.162476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 492/988] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:38.164172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:38.259791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:38.271471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a0a02-25d8-46ea-b5d6-a03a80f91c57 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:38.305866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:38.306527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:38.306809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:38.307076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:38.307910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] RBAC: Adding query filter params (role.id=0c94a10d75ed4085b1f6d83c6c645e15, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=69a0a21790ed496ba0e1e397a9daea09) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:38.308245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:38.309700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16687356-36b5-4a45-8393-c527f7c991e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:38.320108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/989] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:38 2022] GET /identity/v3/role_assignments?user.id=69a0a21790ed496ba0e1e397a9daea09&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=0c94a10d75ed4085b1f6d83c6c645e15 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:38.985612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c379-3f5f-4dfb-b7ab-e726facd4347 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:38.985965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c379-3f5f-4dfb-b7ab-e726facd4347 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:38.986149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9c379-3f5f-4dfb-b7ab-e726facd4347 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:38.987685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 493/990] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:38.996092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:38.996603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:38.997080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.017915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:39.019142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:39.098862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/991] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:38 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:39.122004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9c379-3f5f-4dfb-b7ab-e726facd4347 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.232240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9c379-3f5f-4dfb-b7ab-e726facd4347 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.234601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:39.269179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.269892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.270146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.270432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.271073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.272497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:39.284456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:39.285498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.291496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 494/992] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:39.294067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.387963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:39.406051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c150a2b3-e982-4c76-8865-8cd578485282 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.444284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.444981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.445259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.445536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.446271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:39.446577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.447979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.453939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/993] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:39.460073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.568861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d745843-dcf8-4f40-b19e-83d44f88bc74 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.607849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.608535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.608898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.609163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.609872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:39.610581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.611969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b768b46b-8821-469a-bd42-3f74c66f2410 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.617187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 495/994] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:39.623106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.689071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719fc9c-c645-4bff-97a9-4054b12c8ae8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.689071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719fc9c-c645-4bff-97a9-4054b12c8ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.689271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719fc9c-c645-4bff-97a9-4054b12c8ae8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.690550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 496/995] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:39.695775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.696087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.696366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.713523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:39.715049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:39.736610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36db4e-5708-4d7d-9dc8-bf28b1024635 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.744466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:39.771713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.772433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.772709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.772953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.773735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] RBAC: Adding query filter params (interface=public, service_id=89ec5c1821324ab0a1555af47c21a847, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:39.774068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.775455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.780545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/996] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/endpoints?service_id=89ec5c1821324ab0a1555af47c21a847&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:39.786240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.791298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 497/997] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:39.801642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:39.814865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:39.885354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c71d2-175d-4832-856f-3fe1ecfcbab8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.917452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057923-1c9f-4737-bd17-77eb8dc5272b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:39.920111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.920785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.921064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.921357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.922026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.923519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.930610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/998] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services => generated 1727 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:25:39.956763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:39.958011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:39.958491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:39.958928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:39.959771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:39.961412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:39.975153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:39.976266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:25:39.977114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:39.977847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 498/999] 10.222.0.55 () {56 vars in 1126 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:39.984518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:40.075155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:40.102025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fac1687-466c-49e7-9249-37ef01267f1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:40.144176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:40.145148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.145619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.146072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.146961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:40.147261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:40.148680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:40.154139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/1000] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:40.160197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:40.193350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:40.269724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83a6da-adff-4338-956d-e0d86054fde3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:40.304781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:40.305447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.305721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.305985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.306682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:40.306978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:40.308393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28445598-9f5e-4124-a3ee-4f514eac4669 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:40.317925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 499/1001] 10.222.0.55 () {56 vars in 1122 bytes} [Tue May 24 15:25:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:40.319437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:40.431848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a32f1-7a19-41c6-aca8-571af24c380d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:40.466635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:40.467294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.467570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.467844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.468622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:40.470044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080894c-61a3-4224-a7e3-a6238987639e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:40.482935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:40.496422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/1002] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:40.585652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419680-65f4-4374-b0b1-d3aca4d2e32e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.585990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419680-65f4-4374-b0b1-d3aca4d2e32e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.586242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff419680-65f4-4374-b0b1-d3aca4d2e32e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.587707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 500/1003] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:40.593678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.594020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.594218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.613291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:40.614510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:40.707930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/1004] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:40.732784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff419680-65f4-4374-b0b1-d3aca4d2e32e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:40.833683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:40.845570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff419680-65f4-4374-b0b1-d3aca4d2e32e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:40.880450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:40.881249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:40.881579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:40.881848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:40.882508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:40.883876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:40.896183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:40.897201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:40.898058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 501/1005] 10.222.0.55 () {56 vars in 1105 bytes} [Tue May 24 15:25:40 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:40.905757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:41.007950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb55d9-3be5-40ab-8751-60447aa4b7f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:41.034134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:41.043026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:41.044022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:41.044334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:41.044614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:41.045332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:41.045643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:41.047531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857bf85-84ef-4157-aec4-7aabd01ebd1d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:41.052997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/1006] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:41.058539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:41.176085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eae34b-d3b9-4970-b266-ae806f9bf7cc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:41.221964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:41.222871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:41.223247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:41.223611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:41.224641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:41.225060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:41.227080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c18bf-816f-4fad-bf5c-ff31db376e7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:41.240081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 502/1007] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:42.233372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd0b0-40cc-4b16-b202-7384abd66a56 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.233966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd0b0-40cc-4b16-b202-7384abd66a56 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.234044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fd0b0-40cc-4b16-b202-7384abd66a56 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.235574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/1008] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:42.240889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.241313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.241481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.263569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:42.263569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:42.289024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:42.349632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 503/1009] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:42 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:42.372947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd0b0-40cc-4b16-b202-7384abd66a56 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:42.488949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fd0b0-40cc-4b16-b202-7384abd66a56 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:42.526506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:42.527331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.527605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.527872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.528649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:42.530090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:42.542906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:42.544457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:42.545357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/1010] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:42.552876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:42.654430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29305f04-d20d-4b78-aa66-9f69838e3b77 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:42.702339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:42.703017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.703293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.703564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.704618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:42.704925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:42.706330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9550-6f19-416f-b745-912fd5a824f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:42.711528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 504/1011] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:42.717202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:42.729337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a53be6-9c1a-42d4-809f-3bdd05f8bb1f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.729633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a53be6-9c1a-42d4-809f-3bdd05f8bb1f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.729912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a53be6-9c1a-42d4-809f-3bdd05f8bb1f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.731338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 505/1012] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:42.737080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.737520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.737915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.755968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:42.757142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:42.765972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:42.787115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:42.835807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 506/1013] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:42 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:42.836011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7b772-a07f-4f35-be85-33920086c8a2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:42.859004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:42.873616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:42.874565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:42.874565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:42.874838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:42.875597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:42.875905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:42.877440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:42.887411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/1014] 10.222.0.55 () {56 vars in 1112 bytes} [Tue May 24 15:25:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:42.888280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:42.972580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956101a-99e2-40b8-99a2-a71b24cf0444 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:43.000517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2066d17-956c-4710-95b4-cd0ea758fe8c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:43.012526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:43.013178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.013457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.013731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.014921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:43.016599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:43.018471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 507/1015] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:43.019518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:43.024210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:43.037381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:43.037759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:43.038747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.039048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.039405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.040127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:43.041463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1c76f-7f18-41de-b2b1-a3300a4c501e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:43.063258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/1016] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:43.147530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dac77-5001-4115-9a1e-8e05a16665f1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:43.191798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:43.192496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.192776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.193049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.193757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:43.195211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04005df3-6f32-46dd-b183-ad3e3a950be8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:43.225060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 508/1017] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:43 2022] POST /identity/v3/projects => generated 311 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:25:43.524642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adfc8-989c-43a2-9aeb-69c150192df9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.525160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adfc8-989c-43a2-9aeb-69c150192df9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.525268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534adfc8-989c-43a2-9aeb-69c150192df9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.526746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/1018] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:43.532747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef8677-79d7-4d83-94ad-b0605bf6ac73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.533181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef8677-79d7-4d83-94ad-b0605bf6ac73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.533350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef8677-79d7-4d83-94ad-b0605bf6ac73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.551506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-73ef8677-79d7-4d83-94ad-b0605bf6ac73 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:43.552739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73ef8677-79d7-4d83-94ad-b0605bf6ac73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:43.611927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:43.630836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 509/1019] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:43 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:43.654551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534adfc8-989c-43a2-9aeb-69c150192df9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:43.782156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534adfc8-989c-43a2-9aeb-69c150192df9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:43.822538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:43.823212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.823493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.823764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.824527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:43.825903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fc141-8fa6-476e-b35f-65b88b7c4a70 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:43.840877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/1020] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:25:43 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:45.491646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de940-fea1-4448-a184-d91bcb5c58db None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.492335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de940-fea1-4448-a184-d91bcb5c58db None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.492335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9de940-fea1-4448-a184-d91bcb5c58db None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.493803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 510/1021] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:45.498834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.499241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.499417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.515980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:45.517342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:45.545055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:45.566530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5504e-47c0-404d-99d4-b9746610ed92 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.567036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5504e-47c0-404d-99d4-b9746610ed92 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.567316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5504e-47c0-404d-99d4-b9746610ed92 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.569243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 511/1022] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:45.574225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.574509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.574782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.591836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:45.593092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:45.596286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1023] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:45.616993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:45.670295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 512/1024] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:45.695580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:45.729632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f4d91-efa3-440f-90f0-68aee6e77571 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:45.767990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:45.769622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.769902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.770174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.787421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:45.795992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:45.796999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:45.798718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1025] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:45.803219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:45.805450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703924a-6467-4f60-b633-b7367031fc09 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:45.825189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:45.826479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:45.845403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:45.846226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.846505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.846779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.847980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:45.849483 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:45.851121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 513/1026] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:45.856823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:45.922454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d33a8f5-481a-4d3c-a238-98ccfc33475c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:45.965418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d1dc5-81d5-4263-9a75-6349f28cfa0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:45.965964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:45.966641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:45.967079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:45.967368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:45.968834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:45.969156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:45.970781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:45.984272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1027] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:45.985558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.006775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.007455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.007737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.008010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.008750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.010187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e052de9-7c7d-41a4-a38a-d13af6424f3d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.044147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 514/1028] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:25:45 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:46.077258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.091427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15336ee5-cd90-41a6-bf9a-5caffcf6b2cb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.091778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15336ee5-cd90-41a6-bf9a-5caffcf6b2cb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.091981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15336ee5-cd90-41a6-bf9a-5caffcf6b2cb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.093446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 515/1029] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:46.100296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1046e3a6-c289-4176-852e-3f5fc3a1262b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.101389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.101706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.102001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.123339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:46.125726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:46.144915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.145742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.146023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.146295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.159463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.161325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.178585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:46.181419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:46.182320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:46.183614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:46.183614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:46.183614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:46.183614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:46.183614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1030] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:46.189236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.218110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 516/1031] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:46.240652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.301390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a908bf3-e1d5-4697-aa80-7a56639f77dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.335623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.336333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.336612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.336883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.337607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:46.337911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.339295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.340918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.352780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1032] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:46.357864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.364951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcf3b3-3e4a-466b-b1fa-0e74699e0198 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.405014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.406017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.406668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.406955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.407959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.409716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:46.425431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:46.426516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:46.427416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 517/1033] 10.222.0.55 () {56 vars in 1105 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:46.432832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.471387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.476217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154a6f3-78e7-4893-99d5-4c8c0359fe2a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.511604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.512523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.512868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.513208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.520619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.522154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:46.536865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:46.538187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:46.539345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:46.544933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1034] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:46.545647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.548281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b7a03-62f6-48b8-9121-dba8aaa91f23 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.582193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.582885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.583224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.583556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.584358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:46.584662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.586031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.591213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 518/1035] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:46.596839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.605567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.655076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c35657-cbca-4a20-b568-ab3e681450e8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.698512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.701517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a5021-2407-4fd2-a6a7-32181bb6041f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.701818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.702498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.702780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.703052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.705133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:46.705455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.706946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.728067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1036] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:46.733705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.742920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.743603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.743883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.744194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.744975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:46.745203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.746649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.752885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 519/1037] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:46.758561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.845014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce8b2a-9612-411e-8211-505d28378d24 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.858255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.865595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472de94-ac44-434a-a119-fedb99166ef9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:46.883912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.884605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.884890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.885172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.900061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:46.900736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:46.901012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:46.901282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:46.902075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] RBAC: Adding query filter params (interface=public, service_id=a0624ecb0afe4c8db8a4a89c775d6a45, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:46.902387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.903760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:46.909394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 520/1038] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/endpoints?service_id=a0624ecb0afe4c8db8a4a89c775d6a45&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:46.914538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:46.944064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:46.961769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:46.963237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.019978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5050ad1-561e-4b99-9e6b-6ca8a44782eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.053554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.054219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.054502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.054777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.055461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.056874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.063524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 521/1039] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/services => generated 1957 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:25:47.066067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1040] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:46 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:25:47.071997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:47.177164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:47.187946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a634c883-07d8-45dd-81de-1a0be0e067fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.228609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.229438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.229718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.229988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.242644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.244136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:47.263320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:47.264327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:47.265236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:47.265236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:47.265236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:47.265236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:47.265236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 522/1041] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:47.270424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:47.288067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:47.373386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981dda6-bdc6-4e7c-87d1-c0efc057cb6f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.407766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.408457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.408711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.409010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.409730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:47.410035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.411423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.423667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1042] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:47.429187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:47.533575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1235a-fffd-4689-915c-f0fbe1f71321 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.567003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.567801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.568103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.568379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.575039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.576520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:47.591471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:47.592680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:47.593647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:47.594641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 523/1043] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:47.598643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:47.676098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:47.711782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b38a7f-93dc-434b-9e02-c4c51cf8bcee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.746262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.746899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.747196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.747464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.748266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:47.748573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.750011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad755a1-f1e2-4bdb-be9e-7263c60ab67f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.762909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1044] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:47.769161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:47.774061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:47.874300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243df9c4-b2cc-4788-9748-2f498d1ba8ea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:47.909357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:47.910011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:47.910286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:47.910555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:47.911377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] RBAC: Adding query filter params (role.id=0c94a10d75ed4085b1f6d83c6c645e15, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:47.911685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:47.913148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0883490-be1a-4f38-9ada-15b3dc95b852 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:47.919358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 524/1045] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:47 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=0c94a10d75ed4085b1f6d83c6c645e15 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:48.291177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7658e9-aca7-44ab-a0b8-13ef5381bfd2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:48.291588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7658e9-aca7-44ab-a0b8-13ef5381bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:48.291691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7658e9-aca7-44ab-a0b8-13ef5381bfd2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:48.293616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1046] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:48.299045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:48.299409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:48.299753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:48.317033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:48.318231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:48.387142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:48.394951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 525/1047] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:48.418174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7658e9-aca7-44ab-a0b8-13ef5381bfd2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:48.525295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7658e9-aca7-44ab-a0b8-13ef5381bfd2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:48.554945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:48.561237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:48.562038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:48.562310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:48.562578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:48.572856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:48.580566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:48.581497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:48.587484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1048] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:48.588268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:48.702399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be0844-07ec-42da-92f6-ca103d8ea62e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:48.738116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:48.738783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:48.739043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:48.739330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:48.740064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:48.740384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:48.741753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:48.750089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 526/1049] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:48.755913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:48.857094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae621fe-de11-45a9-9322-73b2b716ed82 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:48.891327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:48.892226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:48.892503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:48.892757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] PATH_INFO: `/v3/users/6bd3983be36241099d1867427dd3bf3f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:48.907084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd3983be36241099d1867427dd3bf3f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:48.908562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:48.910392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1050] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:25:48 2022] GET /identity/v3/users/6bd3983be36241099d1867427dd3bf3f => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:48.915891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:48.975105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:49.018991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e53551-0b27-437f-934f-dbc28510bc93 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.053172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.053819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.054094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.054363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.122250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.123707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b549-2043-4600-b14a-863324104f3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.184438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:49.226790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 527/1051] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:49.232485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:49.283525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:49.315531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68eba4-4bd6-40be-93b4-40212068792a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.315948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68eba4-4bd6-40be-93b4-40212068792a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.316257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68eba4-4bd6-40be-93b4-40212068792a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.317858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 528/1052] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:49.322622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.322906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.323179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.338134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41db75-a9a4-491d-ba8c-d55465b622a2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.340233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:49.341395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:49.376692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.377468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.377742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.378009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.379169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.380664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.382335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1053] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:25:49 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:49.388258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:49.415482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 529/1054] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:49.438429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:49.490293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac708b2-a8ca-4787-b154-aa207cfb0e2c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.508099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:49.526097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.526735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.527016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.527284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.528125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user.id=6bd3983be36241099d1867427dd3bf3f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:49.528430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.529872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.535499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1055] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:49 2022] GET /identity/v3/role_assignments?user.id=6bd3983be36241099d1867427dd3bf3f&scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:49.537751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:49.549566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5c08f-8b2e-4699-8939-8d73e354502f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.590129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.591736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.592193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.592611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.593467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.595511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:49.608610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:49.609796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:25:49.610738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:49.616925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 530/1056] 10.222.0.55 () {56 vars in 1105 bytes} [Tue May 24 15:25:49 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:49.617141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:49.737985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9130ac-303d-4eb9-b944-e926de798224 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.773179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.773844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.774085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.774370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.775066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] RBAC: Adding query filter params (name=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:49.775362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.776761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.781964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1057] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:49.787570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:49.897685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702bc2-8bae-42b7-9bb7-af457da2b040 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:49.935057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:49.935721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:49.935996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:49.936296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:49.937000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:49.937300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:49.938950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5423291-8054-45d6-a48c-d5e500964fec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:49.944021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 531/1058] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:49.949591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.000185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:50.059136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862af85-f7d1-4566-b758-12da3130410c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:50.079649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4718431-6e28-4e1b-aae0-9e82f6301430 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.079939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4718431-6e28-4e1b-aae0-9e82f6301430 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.080240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4718431-6e28-4e1b-aae0-9e82f6301430 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.081745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 532/1059] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:50.086820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.087104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.087382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.096627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:50.097534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.097991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.098460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.099865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:50.101779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:50.104208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:50.105388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:50.120068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:50.132058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1060] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:25:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:25:50.196145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 533/1061] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:50.215506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.329898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90510f7d-b118-477a-8df6-16f5560cbd86 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:50.334401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:50.371828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:50.372711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.372985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.373249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.383445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:50.392440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:50.393671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:50.394550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1062] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:50.399713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.500337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99518c60-b131-4498-a5d4-c84544dfdf6d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:50.533895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:50.534563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.534836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.535104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.535836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:50.536169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:50.537531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:50.546225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 534/1063] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:50.552059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.653981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b49fd5-fc84-47b9-b7b0-b62274e6adb6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:50.698585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:50.699383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.699748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.700052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.711411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:50.713092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:50.730602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:50.731661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:50.732623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:50.732623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:50.732623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:50.732623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:50.732623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1064] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:50.737709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.750096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:50.844554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c0931-742d-4572-b171-1d775541bb05 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:50.877725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:50.878378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:50.878657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:50.878925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:50.879634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:50.879941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:50.881336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:50.894106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 535/1065] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:50.899574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:50.916682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.004425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f99f6-b69f-4d28-82a6-64ee86f9e17a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.038290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.039083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.039356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.039621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.047623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.049093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:51.062987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:51.064061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:51.065067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1066] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:51.070194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.134564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.171676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14918bf2-d321-4402-8513-e868d6799a24 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.207261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.207922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.208228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.208496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.209325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:51.209628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.211055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.232427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 536/1067] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:51.233984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.312234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.339507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bd8f3-407c-4e05-8352-a1e6d4bd5656 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.373929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.374730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.375006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.375344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.386423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.387837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:51.405834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:51.407079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:51.408007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:51.408007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:51.408007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:51.408007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:51.408007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1068] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:51.412738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.518651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0aa27-885e-4f53-8c67-4b54a24c1d16 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.545535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.559152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.560216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.560668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.561110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.562023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:51.562500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.564253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d67c5-ebcf-4336-a985-bf173a851b85 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.582547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 537/1069] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:51.583894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.695555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e22d9-3838-4640-b6b5-b73ceeb1e362 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.722556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.732258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.733057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.733340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.733605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.740608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.744467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.744592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd02d-60e4-43ea-a16d-685a6b8bb8c7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.744592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd02d-60e4-43ea-a16d-685a6b8bb8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.744592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd02d-60e4-43ea-a16d-685a6b8bb8c7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.746620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 538/1070] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:51.751909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.752223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.752501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:51.762051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:51.762962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:51.764090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1071] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:51.768986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.775820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:51.777052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:51.857769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 539/1072] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:51.879135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a78d4-8950-4bab-8d31-72fc8f5e621c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:51.880013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.914523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:51.915205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:51.915463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:51.915734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:51.916547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:51.916853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:51.918620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:51.930526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1073] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:51.936436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:51.984998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:51.989901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd185b47-1c31-4d4d-a5de-4fc9cc7594d3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.004906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:52.027422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.028255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.028533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.028810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.040538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:52.043166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de28d-9e68-4ad2-b34b-757184fb4d68 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:52.048546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:52.049504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:52.050386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:52.054830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 540/1074] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:52.055523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:52.077813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.078770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.079047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.079315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/0c94a10d75ed4085b1f6d83c6c645e15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.081560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=c87b5039c28044988e7d8c418c286994, role_id=0c94a10d75ed4085b1f6d83c6c645e15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.082998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.098407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1075] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:51 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/c87b5039c28044988e7d8c418c286994/roles/0c94a10d75ed4085b1f6d83c6c645e15 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:25:52.176171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7fd104-be5f-4875-89d8-19e6059211be None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.218242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.219274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.219694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.220143 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.221044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:52.221491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.223059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274629c-e794-4f92-ad5a-92a5aee23a48 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.238864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 541/1076] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:52.240238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:52.348424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df7c605-7e02-4bb4-bf7b-b5e365a9b9e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.355551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:52.383624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069b44-c756-4162-a553-840997d0e145 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.384007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069b44-c756-4162-a553-840997d0e145 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.384417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd069b44-c756-4162-a553-840997d0e145 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.385405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.385937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 542/1077] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:52.386282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.386554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.386821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] PATH_INFO: `/v3/users/6bd3983be36241099d1867427dd3bf3f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.387997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd3983be36241099d1867427dd3bf3f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.389667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.391333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1078] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:25:52 2022] GET /identity/v3/users/6bd3983be36241099d1867427dd3bf3f => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:52.391546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.391637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.392002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.396779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:52.409825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:52.411251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:52.442190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:52.488108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 543/1079] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:52.503655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5d105-349e-4c29-84b4-6a3f79a320f1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.508774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:52.541336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.542200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.542473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.543045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.544231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.545669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.547963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1080] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:25:52 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:52.553566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:52.597603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:52.620677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708e310-0e3c-4db6-92df-199a03172897 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.656158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed108d8-b082-4e85-9fe7-7b4f137b7b4f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:52.657345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.658008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.658289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.658564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.697139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:52.699456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:52.700843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:52.701152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:52.701713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4/users/6bd3983be36241099d1867427dd3bf3f/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:52.703924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user_id=6bd3983be36241099d1867427dd3bf3f, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.705413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.720082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1081] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:25:52 2022] PUT /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4/users/6bd3983be36241099d1867427dd3bf3f/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:25:52.737093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:52.738900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:52.859365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 544/1082] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:25:52.870168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:53.008549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b30e-712b-495d-aa5d-c7dba7b14241 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:53.010229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:53.050615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:53.051422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:53.051703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:53.051966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:53.065167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:53.066633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-34524568-67a7-41b8-b4f1-eacacfb90d9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:53.084424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:53.085371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:53.086283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:53.086283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:53.086283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:53.086283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:53.086283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1083] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:53.091802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:53.163883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:53.202071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9a913-bef2-481a-9279-aae827e2c184 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:53.238625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:53.239308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:53.239584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:53.239856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:53.240613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:53.240912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:53.242285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d250a32e-7adc-47b1-b65a-dc5acee95da5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:53.254365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 545/1084] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:54.293446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e6bd-6dec-4835-b178-1d1429eee654 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.294011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e6bd-6dec-4835-b178-1d1429eee654 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.294011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518e6bd-6dec-4835-b178-1d1429eee654 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.295343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1085] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:54.300936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.301383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.301473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.318582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:54.319757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:54.388055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 546/1086] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:54.411278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e6bd-6dec-4835-b178-1d1429eee654 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:54.522981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518e6bd-6dec-4835-b178-1d1429eee654 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:54.559017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:54.559739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.560017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.560312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.570512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:54.578771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:54.579724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:54.580682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1087] 10.222.0.55 () {56 vars in 1120 bytes} [Tue May 24 15:25:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:54.586169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:54.644524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:54.690778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96152712-4cd3-4306-a14b-495986a1b8e2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:54.725887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:54.726560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.726839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.727112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.727854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:54.728197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:54.729570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:54.737623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 547/1088] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:25:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:54.743478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:54.760179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:54.851354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe577-8915-4da7-a47f-bc46c9007038 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:54.885900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:54.886692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.886971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.887240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.898187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:54.899643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5065d937-73b5-4145-895f-a44db6e568f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:54.916812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:54.918037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:54.918941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:54.918941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:54.918941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:54.918941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:54.920218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1089] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:54.924130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:54.986974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042112bb-0114-4377-a0df-f46662cae0db None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.987264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042112bb-0114-4377-a0df-f46662cae0db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.987541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042112bb-0114-4377-a0df-f46662cae0db None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:54.989127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1090] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:25:54.993910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:54.994170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:54.994481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.011063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:55.012295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:55.028613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558dd57-b2dc-448f-9d26-5fb28ee7cc00 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.045629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.062783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.063540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.063990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.064118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.064808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:55.065109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.066469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.074392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.078468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 548/1091] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:55.084009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.086703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1092] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:25:55.113155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.189457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a94ce-50ca-4ccb-9d3b-efcd4b959593 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.212541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33bb0f-c21e-4232-85f6-a8f6818a84c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.226217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.227017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.227297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.227572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.235346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.236841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.246304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.247096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.247372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.247641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:25:55.251534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:55.252570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:55.253622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 549/1093] 10.222.0.55 () {56 vars in 1093 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:55.258445 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.259173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:25:55.266765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:55.267915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:25:55.268819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1094] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:55.273859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.301255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.330762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.363366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163b772-2f92-4c86-bf0e-c6f4c37a0ccf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.373372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb66afcf-5116-41f5-8418-8af4bab8af1f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.397907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.398576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.398859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.399110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.399948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:55.400295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.401738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.407485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.408197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.408493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.408754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.409495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:55.409802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.411549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.420235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 550/1095] 10.222.0.55 () {56 vars in 1133 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:55.424905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1096] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:55.425957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.426078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.517171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.523568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559a1be-7846-42d3-9f0a-3a475fe1f5f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.523710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1ae65-4ba8-428d-baf4-450cdac3f2a4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.541807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.558942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.559586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.559860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.560162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.560795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.561689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.561972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.562250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] PATH_INFO: `/v3/users/6bd3983be36241099d1867427dd3bf3f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.563749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd3983be36241099d1867427dd3bf3f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.565394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.567044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1097] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/users/6bd3983be36241099d1867427dd3bf3f => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:55.572749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.629613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.630999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.671664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602acb6-0566-427d-8792-eddf35988b02 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.705456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.706107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.706382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.706651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.726803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.734718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 551/1098] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:55.740532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.760801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.775113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.776632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.838242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756712a-23c8-4f30-b896-606be33ad160 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:55.872945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:55.873746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:55.874018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:55.874282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:55.875956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1099] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:25:55.882253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.885217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:55.886690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:25:55.904336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:55.905256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:55.906164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:25:55.906164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:25:55.906164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:25:55.906164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:55.906164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 552/1100] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:25:55.911744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:55.960060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.967086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:55.982746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad92a7-542c-4d34-8ea8-911fb59b4bf4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.012146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d79892-671b-4154-a02a-a442db6bde27 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.015445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.016600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.016876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.017171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.018332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.019766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.021501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1101] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:56.027429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:56.047631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.048322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.048602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.048877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.049625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:56.049925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.051291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.062600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 553/1102] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:56.068121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:56.130879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699be709-87a7-4878-aaa2-b258468efabd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.164655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.165123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3411e2-78b3-40fa-8944-02b1a434c1b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.165246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.165493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.165763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.166580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user.id=6bd3983be36241099d1867427dd3bf3f) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:56.166885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.168661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.174458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1103] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:56 2022] GET /identity/v3/role_assignments?user.id=6bd3983be36241099d1867427dd3bf3f&scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:25:56.175122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:56.204322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.205156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.205430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.205694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.213963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.215449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:25:56.231473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:25:56.232829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:25:56.234249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 554/1104] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:25:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:25:56.238879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:56.260125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:56.347911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6d0b7-13d2-4b3a-a6a4-61642c983d55 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.385319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.385988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.386264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.386534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.387360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:56.387698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.389188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73d0b5-7591-4123-b92f-0991e25f9f23 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.401803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1105] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:25:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:56.408467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:56.509353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfeb542-c0ed-43d9-8489-6d344451a3b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:56.544699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:56.545354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:56.545635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:56.545909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:56.546734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] RBAC: Adding query filter params (role.id=0c94a10d75ed4085b1f6d83c6c645e15, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=c87b5039c28044988e7d8c418c286994) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:25:56.547045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:56.548858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7a5f4d-195b-4a0e-9fc9-af80b03301a5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:56.555004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 555/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:25:56 2022] GET /identity/v3/role_assignments?user.id=c87b5039c28044988e7d8c418c286994&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=0c94a10d75ed4085b1f6d83c6c645e15 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:58.371451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785022-5fc7-4df2-a5e2-b6cde1ce09a8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:58.372565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785022-5fc7-4df2-a5e2-b6cde1ce09a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:58.372565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62785022-5fc7-4df2-a5e2-b6cde1ce09a8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:58.374564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1107] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:25:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:25:58.383319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140b32-7191-4814-be37-8426f9428f10 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:58.383745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140b32-7191-4814-be37-8426f9428f10 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:58.383908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140b32-7191-4814-be37-8426f9428f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:58.404232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-84140b32-7191-4814-be37-8426f9428f10 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:25:58.405410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84140b32-7191-4814-be37-8426f9428f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:58.432681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:58.478386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 556/1108] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:25:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:25:58.505371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62785022-5fc7-4df2-a5e2-b6cde1ce09a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:58.592711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:58.628822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62785022-5fc7-4df2-a5e2-b6cde1ce09a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:58.666446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:58.667232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:58.667806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:58.668084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:58.669382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:58.670823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2c87c-2e61-4be8-bd51-e385b2e2bbe8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:58.672711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:25:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:25:58.681968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84140b32-7191-4814-be37-8426f9428f10 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:58.800046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84140b32-7191-4814-be37-8426f9428f10 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:58.835230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:58.835887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:58.836193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:58.836465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:58.837185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:58.838634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ecc02-8323-4326-87a4-9bc8a99fe39d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:58.864945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 557/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:25:58 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:01.070129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b391-01da-49e9-b259-42aa66cf4ab2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:01.071008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b391-01da-49e9-b259-42aa66cf4ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:01.071008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6b391-01da-49e9-b259-42aa66cf4ab2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:01.073234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1111] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:01.081698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:01.082129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:01.082216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:01.100258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:01.101433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:01.114644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:01.179048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 558/1112] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:26:01.212875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6b391-01da-49e9-b259-42aa66cf4ab2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:01.331853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6b391-01da-49e9-b259-42aa66cf4ab2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:01.367195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:01.368076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:01.368353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:01.368698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] PATH_INFO: `/v3/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:01.370192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] RBAC: Authorizing `identity:get_role(role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:01.371576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:01.373015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:01 2022] GET /identity/v3/roles/85a7aff706174bb09e06d1ca1d363973 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:01.379280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:01.482481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41623429-bff8-4b6d-b434-8b9d6d60692d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:01.516594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:01.517451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:01.517720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:01.517985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] PATH_INFO: `/v3/users/755ecaa3370f44b3a6b14d217a8aa286` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:01.531120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] RBAC: Authorizing `identity:get_user(user_id=755ecaa3370f44b3a6b14d217a8aa286)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:01.532593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:01.534208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 559/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:01 2022] GET /identity/v3/users/755ecaa3370f44b3a6b14d217a8aa286 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:01.540233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:01.556989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:01.657662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b29408-a34e-4983-b93e-911034db2dbc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:01.693105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:01.693771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:01.694042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:01.694316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:01.763137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:01.764625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:01.780595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:01.873854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:01.879938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:01.889317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:01.986966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea6606-cc4d-4eee-bc81-14255b25964c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:02.022267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:02.023110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:02.023383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:02.023651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:02.024962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:02.026387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fa826-2602-471e-9a58-d7e419c4fb83 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:02.028059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 560/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:01 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:02.034533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:02.136169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06c2bf-703c-4c87-b1e0-e3332af9eeee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:02.150097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:02.172822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:02.173459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:02.173736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:02.174007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:02.174910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user.id=755ecaa3370f44b3a6b14d217a8aa286) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:02.175216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:02.176788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdd5b8-7dca-4b45-8032-f4894531807d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:02.182811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:02 2022] GET /identity/v3/role_assignments?user.id=755ecaa3370f44b3a6b14d217a8aa286&scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:04.313823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cba921-2bc8-455b-b1dc-48e581b57c54 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:04.314668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cba921-2bc8-455b-b1dc-48e581b57c54 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:04.314668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cba921-2bc8-455b-b1dc-48e581b57c54 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:04.317000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 561/1118] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:04.323279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:04.323741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:04.323833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:04.344332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:04.345516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:04.416906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1119] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:04.441613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cba921-2bc8-455b-b1dc-48e581b57c54 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:04.523697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:04.551427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cba921-2bc8-455b-b1dc-48e581b57c54 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:04.584404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:04.585252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:04.585526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:04.585790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] PATH_INFO: `/v3/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:04.587340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] RBAC: Authorizing `identity:get_role(role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:04.588750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:04.590142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 562/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:04 2022] GET /identity/v3/roles/85a7aff706174bb09e06d1ca1d363973 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:04.596280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:04.712481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2353e-51ec-41f8-8f9d-796f4ce2eff8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:04.721920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:04.748633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:04.749506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:04.749857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:04.750137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] PATH_INFO: `/v3/users/755ecaa3370f44b3a6b14d217a8aa286` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:04.751444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] RBAC: Authorizing `identity:get_user(user_id=755ecaa3370f44b3a6b14d217a8aa286)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:04.752908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:04.754559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:04 2022] GET /identity/v3/users/755ecaa3370f44b3a6b14d217a8aa286 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:26:04.760629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:04.864182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f413-8c99-4ca8-86ca-c707642ad60a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:04.899303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:04.900189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:04.900475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:04.900747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:04.902129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:04.903573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b9184-b2ee-4b3f-a7d8-59a56aeb6641 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:04.905387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 563/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:04 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:04.912020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:05.019438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b0f00-a3cd-410f-81c8-4aae969c4c9e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:05.054543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:05.055422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:05.055690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:05.055960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4/users/755ecaa3370f44b3a6b14d217a8aa286/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:05.058551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user_id=755ecaa3370f44b3a6b14d217a8aa286, role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:05.059978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429ff96-b70f-454c-8edb-8b5cf43fca73 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:05.080432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:26:04 2022] PUT /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4/users/755ecaa3370f44b3a6b14d217a8aa286/roles/85a7aff706174bb09e06d1ca1d363973 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:26:07.223818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d09c7-6f0e-4cf8-b6be-053bac3ba13d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:07.224436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d09c7-6f0e-4cf8-b6be-053bac3ba13d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:07.224436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d09c7-6f0e-4cf8-b6be-053bac3ba13d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:07.225742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 564/1124] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:07.232254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:07.232698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:07.232828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:07.260334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:07.262216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:07.298043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:07.340079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1125] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:07.363946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d09c7-6f0e-4cf8-b6be-053bac3ba13d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:07.383215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:07.481119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d09c7-6f0e-4cf8-b6be-053bac3ba13d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:07.515787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:07.516677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:07.516956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:07.517221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] PATH_INFO: `/v3/roles/85a7aff706174bb09e06d1ca1d363973` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:07.518792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] RBAC: Authorizing `identity:get_role(role_id=85a7aff706174bb09e06d1ca1d363973)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:07.520192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:07.521546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 565/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:07 2022] GET /identity/v3/roles/85a7aff706174bb09e06d1ca1d363973 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:07.528201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:07.639368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded14c0a-f57d-486e-90a0-95cd88019467 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:07.676893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:07.677767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:07.678038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:07.678288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] PATH_INFO: `/v3/users/755ecaa3370f44b3a6b14d217a8aa286` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:07.679482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] RBAC: Authorizing `identity:get_user(user_id=755ecaa3370f44b3a6b14d217a8aa286)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:07.680950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:07.682567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:07 2022] GET /identity/v3/users/755ecaa3370f44b3a6b14d217a8aa286 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:07.688713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:07.773424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:07.799735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82493eb0-2dec-40b7-8443-20c568c8eb8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:07.838380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:07.839017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:07.839294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:07.839572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:07.914370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:07.915869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:07.999256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:08.028271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 566/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:08.034476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:08.070433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:08.145431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc46786-deac-4422-83a4-16f9006e6a62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:08.190203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:08.191554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:08.191795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:08.192247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] PATH_INFO: `/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:08.194251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] RBAC: Authorizing `identity:get_project(project_id=ad45b8bd0d6746e6a1f85fe3d9cde7d4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:08.196551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d5e76-a1c2-4cfa-b766-65a5433d69c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:08.199161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:08 2022] GET /identity/v3/projects/ad45b8bd0d6746e6a1f85fe3d9cde7d4 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:08.206786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:08.309833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3a8de-693b-447a-b83e-5c0ccf903623 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:08.345435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:08.346072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:08.346348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:08.346617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:08.347478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] RBAC: Adding query filter params (role.id=85a7aff706174bb09e06d1ca1d363973, scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4, user.id=755ecaa3370f44b3a6b14d217a8aa286) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:08.347789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:08.349262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded93c2-caf9-4c27-98aa-7c0bb05b53c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:08.355171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 567/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:08 2022] GET /identity/v3/role_assignments?user.id=755ecaa3370f44b3a6b14d217a8aa286&scope.project.id=ad45b8bd0d6746e6a1f85fe3d9cde7d4&role.id=85a7aff706174bb09e06d1ca1d363973 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:10.558987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e57adb-120c-4b29-ab01-04bbea659642 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:10.559901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e57adb-120c-4b29-ab01-04bbea659642 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:10.559901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e57adb-120c-4b29-ab01-04bbea659642 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:10.562190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1131] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:10.568316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:10.568767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:10.568859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:10.595670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:10.596895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:10.634339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:10.667532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 568/1132] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:10.690986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e57adb-120c-4b29-ab01-04bbea659642 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:10.797251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e57adb-120c-4b29-ab01-04bbea659642 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:10.798833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:10.836537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:10.837245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:10.837527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:10.837789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:10.838981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:10.840447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f1d2c-6ff6-4fff-aa87-486540d2b0ab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:10.842081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:26:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:10.848120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:10.954369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8133cf69-5ff6-4c79-a44e-7989f7ff9c81 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:10.990691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:10.991331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:10.991603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:10.991867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:10.992601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:10.994027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b4751-d93a-4847-8e26-bde977acad38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:11.023352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 569/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:26:10 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:13.276518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9fb1-e90b-4816-9c87-c8fc6b07fe39 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:13.277373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9fb1-e90b-4816-9c87-c8fc6b07fe39 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:13.277373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8c9fb1-e90b-4816-9c87-c8fc6b07fe39 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:13.279632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1135] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:13.286280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:13.286996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:13.287140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:13.308606 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:13.309800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:13.328272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:13.387602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 570/1136] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:13.412768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c9fb1-e90b-4816-9c87-c8fc6b07fe39 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:13.521291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8c9fb1-e90b-4816-9c87-c8fc6b07fe39 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:13.557883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:13.558600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:13.558879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:13.559152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:13.560388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:13.561834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935d605-6dd2-479c-b334-021c2b0f0d5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:13.563492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:26:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:13.569497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:13.680294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2c91-1e59-438a-9eb3-39c0ab6645da None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:13.718112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:13.718918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:13.719199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:13.719476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:13.720239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:13.721696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49374aec-aa58-4c41-96c0-5bb69cf3283f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:13.745993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 571/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:26:13 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:15.940667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f9e-0a82-4d93-be25-b0bea31f53d9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:15.941218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f9e-0a82-4d93-be25-b0bea31f53d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:15.941218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee9f9e-0a82-4d93-be25-b0bea31f53d9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:15.942519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1139] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:15.948990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:15.949500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:15.949589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:15.974384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:15.976116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:15.979281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:16.049453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 572/1140] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:16.078350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee9f9e-0a82-4d93-be25-b0bea31f53d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:16.129510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:16.191272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee9f9e-0a82-4d93-be25-b0bea31f53d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:16.226233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:16.227012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:16.227285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:16.227547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:16.238438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:16.245879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:16.246955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:16.247851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:16.253176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:16.355528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2da992-34ab-472a-9ee2-a1997ee05467 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:16.391683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:16.392376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:16.392644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:16.393049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:16.393617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:16.393919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:16.395293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:16.403309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 573/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:16.413109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:16.513886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:16.518973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606593d-6552-4fbd-8635-73c0cf604398 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:16.564048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:16.564926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:16.565201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:16.565471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] PATH_INFO: `/v3/users/6933a889c9074b0f8acb4181cfbd6903` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:16.579344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] RBAC: Authorizing `identity:get_user(user_id=6933a889c9074b0f8acb4181cfbd6903)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:16.581757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:16.584489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:16 2022] GET /identity/v3/users/6933a889c9074b0f8acb4181cfbd6903 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:16.593367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:16.596363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:16.708528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521db36-801a-405d-9abc-c85fe902cd7c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:16.745739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:16.746404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:16.746714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:16.746986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:16.816303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:16.817704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:16.819581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:16.920110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 574/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:26:16.926640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:17.043823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231053e-2672-4823-868d-6a7bbe8c5897 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:17.080655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:17.081442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:17.081723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:17.081992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] PATH_INFO: `/v3/projects/6fefdee3f917461d81043b847363fdd8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:17.083295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] RBAC: Authorizing `identity:get_project(project_id=6fefdee3f917461d81043b847363fdd8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:17.084784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483e97c-0a23-48e0-8eb4-d8895eaef46c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:17.086484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:16 2022] GET /identity/v3/projects/6fefdee3f917461d81043b847363fdd8 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:17.093899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:17.201700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79807416-4f04-4a93-8919-05f7e883a8bb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:17.203332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:17.238007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:17.238649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:17.238926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:17.239199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:17.240017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=6fefdee3f917461d81043b847363fdd8, user.id=6933a889c9074b0f8acb4181cfbd6903) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:17.240370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:17.241801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c646b80-b795-4c58-aa6c-9cfd245f1986 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:17.247309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 575/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:17 2022] GET /identity/v3/role_assignments?user.id=6933a889c9074b0f8acb4181cfbd6903&scope.project.id=6fefdee3f917461d81043b847363fdd8&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:19.386449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa71ad-f824-4e12-af6a-43de35514b78 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:19.387008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa71ad-f824-4e12-af6a-43de35514b78 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:19.387008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa71ad-f824-4e12-af6a-43de35514b78 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:19.389377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1147] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:19.395647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:19.396126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:19.396286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:19.414431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:19.415615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:19.488438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 576/1148] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:19.522067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa71ad-f824-4e12-af6a-43de35514b78 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:19.550678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:19.631494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa71ad-f824-4e12-af6a-43de35514b78 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:19.669076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:19.669861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:19.670217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:19.670560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:19.686404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:19.694245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:19.695286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:19.697412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:19.701527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:19.799966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:19.810474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c778530-ccfe-4643-84ae-c83cd83cfb98 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:19.846033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:19.846684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:19.846959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:19.847226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:19.847968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:19.848306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:19.849774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:19.857747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 577/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:19.864169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:19.954506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:19.972103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df86ce-074d-4fca-bcc3-972c5faa4ae2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:20.006692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:20.007544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:20.007822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:20.008085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] PATH_INFO: `/v3/users/6933a889c9074b0f8acb4181cfbd6903` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:20.009350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] RBAC: Authorizing `identity:get_user(user_id=6933a889c9074b0f8acb4181cfbd6903)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:20.010791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:20.012446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:19 2022] GET /identity/v3/users/6933a889c9074b0f8acb4181cfbd6903 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:20.021923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:20.134141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d1fa6-274d-4e99-9266-98499d1e97e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:20.168824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:20.169670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:20.169948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:20.170216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] PATH_INFO: `/v3/projects/6fefdee3f917461d81043b847363fdd8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:20.171485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] RBAC: Authorizing `identity:get_project(project_id=6fefdee3f917461d81043b847363fdd8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:20.172945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aba36a-79ed-4689-9b94-4cb6b0acacdf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:20.174614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 578/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:20 2022] GET /identity/v3/projects/6fefdee3f917461d81043b847363fdd8 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:20.181284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:20.295394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83de7e9-b00e-4766-8fd0-09ec0d37b74a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:20.330577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:20.331468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:20.331745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:20.332018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] PATH_INFO: `/v3/projects/6fefdee3f917461d81043b847363fdd8/users/6933a889c9074b0f8acb4181cfbd6903/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:20.334275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] RBAC: Authorizing `identity:create_grant(project_id=6fefdee3f917461d81043b847363fdd8, user_id=6933a889c9074b0f8acb4181cfbd6903, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:20.335720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795806-a82f-4479-91e8-35f207385790 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:20.356023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:26:20 2022] PUT /identity/v3/projects/6fefdee3f917461d81043b847363fdd8/users/6933a889c9074b0f8acb4181cfbd6903/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:26:22.503841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79180-3be1-4189-8e73-3ad319544fe8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:22.504449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79180-3be1-4189-8e73-3ad319544fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:22.504449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb79180-3be1-4189-8e73-3ad319544fe8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:22.506487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 579/1154] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:22.515724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:22.516157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:22.516378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:22.544573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:22.546421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:22.554899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:22.627217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1155] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:22.651680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb79180-3be1-4189-8e73-3ad319544fe8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:22.684020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:22.765275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb79180-3be1-4189-8e73-3ad319544fe8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:22.801904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:22.802696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:22.802969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:22.803234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:22.814998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:22.822628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:22.823696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:26:22.832125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 580/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:22.832840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:22.938037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b4396-1b5e-497c-9774-2932c7e6eb7b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:22.985738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:22.986393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:22.986672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:22.986938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:22.987681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:22.987983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:22.989390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:22.997727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:23.003820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:23.092228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:23.108070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec518384-39f5-414d-af5c-eeb64d02bd08 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:23.144084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:23.144953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:23.145225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:23.145488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] PATH_INFO: `/v3/users/6933a889c9074b0f8acb4181cfbd6903` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:23.146703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] RBAC: Authorizing `identity:get_user(user_id=6933a889c9074b0f8acb4181cfbd6903)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:23.148165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:23.149772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 581/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:23 2022] GET /identity/v3/users/6933a889c9074b0f8acb4181cfbd6903 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:23.156341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:23.162538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:23.267420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d275b5-be17-4700-bde3-158d0c397d0e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:23.301916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:23.302553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:23.302839 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:23.303075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:23.380589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:23.382004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:23.384779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:23.481559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:23.488409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:23.594969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb88f5-d6a6-4a52-bb12-d1f954aa3e6e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:23.630259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:23.631168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:23.631328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:23.631637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] PATH_INFO: `/v3/projects/6fefdee3f917461d81043b847363fdd8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:23.632952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] RBAC: Authorizing `identity:get_project(project_id=6fefdee3f917461d81043b847363fdd8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:23.634377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe7b78-b3bb-41e1-b97c-d7e06fd24238 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:23.636081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 582/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:23 2022] GET /identity/v3/projects/6fefdee3f917461d81043b847363fdd8 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:23.642353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:23.744251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a035412-a784-4da0-b0f9-4c3489fd9e07 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:23.746635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:23.785291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:23.785934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:23.786206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:23.786472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:23.787293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=6fefdee3f917461d81043b847363fdd8, user.id=6933a889c9074b0f8acb4181cfbd6903) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:23.787597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:23.789071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90889a53-5e93-4710-9c5f-837f55336946 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:23.795305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:23 2022] GET /identity/v3/role_assignments?user.id=6933a889c9074b0f8acb4181cfbd6903&scope.project.id=6fefdee3f917461d81043b847363fdd8&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:26:25.980726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d443-bd61-479f-8196-7a46f068f842 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:25.981386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d443-bd61-479f-8196-7a46f068f842 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:25.981386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d443-bd61-479f-8196-7a46f068f842 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:25.982660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 583/1162] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:25.989213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:25.989974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:25.990110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:26.017342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:26.018533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:26.096955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1163] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:26.119590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d443-bd61-479f-8196-7a46f068f842 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:26.146164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:26.230699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d443-bd61-479f-8196-7a46f068f842 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:26.275826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:26.276432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:26.276711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:26.276980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:26.349815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:26.351264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:26.376367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:26.454485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 584/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:26.464353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:26.575682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:26.579819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39b8c56-fd0c-4e8c-a43f-b6296ad7451e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:26.619734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:26.620628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:26.620903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:26.621172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:26.628061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:26.629547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f909ffef-11f4-49ce-b729-938a388a50f4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:26.644063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:26.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:26.646606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:26:26 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:26.651379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:26.759395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c720cd8-a424-4c2b-a05c-0a2c24e0cf25 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:26.784388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:26.802757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:26.803417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:26.803693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:26.803963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:26.804723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:26.805026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:26.806439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d480cf-623a-41ba-a45a-e8527abdac7b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:26.814234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 585/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Tue May 24 15:26:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:29.001271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3a61-5ad5-4063-bd02-c601104630a5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:29.001853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3a61-5ad5-4063-bd02-c601104630a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:29.001853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf3a61-5ad5-4063-bd02-c601104630a5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:29.003942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1167] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:29.012799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670dfc0-d73d-453e-a420-30296e6acb07 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:29.013210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670dfc0-d73d-453e-a420-30296e6acb07 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:29.013416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670dfc0-d73d-453e-a420-30296e6acb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:29.031871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b670dfc0-d73d-453e-a420-30296e6acb07 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:29.033096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b670dfc0-d73d-453e-a420-30296e6acb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:29.102176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 586/1168] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:29.125274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf3a61-5ad5-4063-bd02-c601104630a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:29.242644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf3a61-5ad5-4063-bd02-c601104630a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:29.286537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:29.287131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:29.287414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:29.287690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:29.288483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:29.289874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc14fb3-a4e8-4e27-86ec-e63b72a5fed0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:29.312183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Tue May 24 15:26:29 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:31.505201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9c061-11c4-4f79-9105-24b6539eb2cc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:31.505798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9c061-11c4-4f79-9105-24b6539eb2cc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:31.505798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9c061-11c4-4f79-9105-24b6539eb2cc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:31.507123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 587/1170] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:31.513542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:31.514264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:31.514408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:31.533196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:31.534375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:31.549504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:31.617216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1171] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:31.642978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9c061-11c4-4f79-9105-24b6539eb2cc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:31.754217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9c061-11c4-4f79-9105-24b6539eb2cc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:31.771025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:31.789853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:31.790710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:31.790982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:31.791249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] PATH_INFO: `/v3/domains/7b38f9daa37a4d60bf0f358c5b4d8295` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:31.802772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7b38f9daa37a4d60bf0f358c5b4d8295)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:31.804246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716fb923-2fc3-4ed0-bf35-032ee01d8358 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:31.805896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 588/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Tue May 24 15:26:31 2022] GET /identity/v3/domains/7b38f9daa37a4d60bf0f358c5b4d8295 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:31.812849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:31.929065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ad170-d134-4465-b58a-cb3e09d727b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:31.964653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:31.965299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:31.965574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:31.965844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:31.966542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:31.968075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee877e9-305c-47db-a6e3-e75ce8d8e213 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:31.992680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:32.001634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:26:31 2022] POST /identity/v3/projects => generated 361 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:34.204503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e275b-8541-4183-afa1-512af423931f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:34.205062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e275b-8541-4183-afa1-512af423931f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:34.205170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e275b-8541-4183-afa1-512af423931f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:34.206534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 589/1174] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:34.212992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:34.213712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:34.213845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:34.238997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:34.240210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:34.310797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1175] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:34.341333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e275b-8541-4183-afa1-512af423931f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:34.449422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e275b-8541-4183-afa1-512af423931f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:34.488498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:34.489652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:34.489918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:34.490280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] PATH_INFO: `/v3/domains/7b38f9daa37a4d60bf0f358c5b4d8295` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:34.492489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7b38f9daa37a4d60bf0f358c5b4d8295)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:34.494757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978e4e0-4a44-4262-829f-36df8f772e88 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:34.497341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 590/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Tue May 24 15:26:34 2022] GET /identity/v3/domains/7b38f9daa37a4d60bf0f358c5b4d8295 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:34.503787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:34.608068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a53fb3-4dbd-4f17-8f89-d8c0389477d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:34.635862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:34.643300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:34.643964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:34.644270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:34.644548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:34.645260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:34.646702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e7312-3af4-42de-8e47-249d0e02c8ef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:34.670320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:26:34 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:26:36.862249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6c90-feb7-4767-8561-011133b05386 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:36.863131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6c90-feb7-4767-8561-011133b05386 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:36.863131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a6c90-feb7-4767-8561-011133b05386 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:36.865400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 591/1178] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:36.871679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:36.872156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:36.872262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:36.890782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:36.891985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:36.963084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1179] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:36.986956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a6c90-feb7-4767-8561-011133b05386 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:37.035372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:37.102896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a6c90-feb7-4767-8561-011133b05386 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:37.139894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:37.140709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:37.140987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:37.141261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:37.153494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:37.161203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:37.162279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:26:37.163237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 592/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:37.169474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:37.270233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e88fd-aa36-4b5d-b06e-4e0c8893fb7f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:37.286712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:37.309533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:37.310200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:37.310476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:37.310747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:37.311676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:37.311990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:37.313398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:37.322375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:37.328563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:37.453288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39d25e-0eed-4765-8b73-08ec0c8f5f8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:37.455623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:37.488131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:37.488990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:37.489259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:37.489520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] PATH_INFO: `/v3/users/5b5ad45215294db0bde0e7bfc7512e02` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:37.502947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] RBAC: Authorizing `identity:get_user(user_id=5b5ad45215294db0bde0e7bfc7512e02)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:37.504411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:37.506041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 593/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:37 2022] GET /identity/v3/users/5b5ad45215294db0bde0e7bfc7512e02 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:37.512686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:37.619230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae61b9-4ee6-4e74-a40d-6a2b7171950e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:37.656811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:37.657441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:37.657716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:37.657983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:37.706406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:37.730583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:37.732072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:37.831760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:37.838826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:37.955099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb600c80-265a-4f60-a32c-1abcb47535fa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:37.989834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:37.990697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:37.990950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:37.991211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] PATH_INFO: `/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:37.992446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] RBAC: Authorizing `identity:get_project(project_id=b8b786e1788045bfb01a4cb3e541d5f4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:37.993879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39640982-6766-4ba5-9d29-6ab618f769fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:37.995554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 594/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:37 2022] GET /identity/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:38.001743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:38.096391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:38.123512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400046d-0f9a-43ab-bde9-ffbf1761c216 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:38.159812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:38.160493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:38.160772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:38.161046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:38.161869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=b8b786e1788045bfb01a4cb3e541d5f4, user.id=5b5ad45215294db0bde0e7bfc7512e02) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:38.162172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:38.163612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600fc3c-7df8-45b2-8e05-97c8325fbdc7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:38.169557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:38 2022] GET /identity/v3/role_assignments?user.id=5b5ad45215294db0bde0e7bfc7512e02&scope.project.id=b8b786e1788045bfb01a4cb3e541d5f4&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:40.312226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271834e2-6dbe-42a7-a6c2-399af5e88867 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:40.313126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271834e2-6dbe-42a7-a6c2-399af5e88867 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:40.313126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271834e2-6dbe-42a7-a6c2-399af5e88867 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:40.315276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 595/1186] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:40.320781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:40.321225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:40.321331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:40.339571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:40.340782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:40.413871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1187] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:40.437201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271834e2-6dbe-42a7-a6c2-399af5e88867 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:40.470513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:40.542811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271834e2-6dbe-42a7-a6c2-399af5e88867 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:40.576902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:40.577611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:40.577885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:40.578156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:40.588364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:40.595945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:40.597017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:26:40.597887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 596/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:40.603958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:40.681726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:40.711735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68372de6-1d09-4548-b93b-f966a258ac66 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:40.746496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:40.747159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:40.747437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:40.747712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:40.748576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:40.748952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:40.750260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:40.758362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:40.765126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:40.870644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:40.882452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2b201-fe43-4adf-a979-135098d05d71 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:40.920535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:40.921419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:40.921694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:40.921942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] PATH_INFO: `/v3/users/5b5ad45215294db0bde0e7bfc7512e02` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:40.923141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] RBAC: Authorizing `identity:get_user(user_id=5b5ad45215294db0bde0e7bfc7512e02)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:40.924599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:40.926227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 597/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:40 2022] GET /identity/v3/users/5b5ad45215294db0bde0e7bfc7512e02 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:40.932291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:41.037873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c623-0129-494a-b45e-473d1e29ed79 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:41.073360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:41.074205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:41.074482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:41.074754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] PATH_INFO: `/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:41.076074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] RBAC: Authorizing `identity:get_project(project_id=b8b786e1788045bfb01a4cb3e541d5f4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:41.077589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbf3ad7-0265-49a6-9d7e-9f2ff57a430c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:41.079288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:40 2022] GET /identity/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:41.085425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:41.187968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e81d4-230f-46c9-b336-d59056f8f15f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:41.223853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:41.224760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:41.225029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:41.225292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] PATH_INFO: `/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4/users/5b5ad45215294db0bde0e7bfc7512e02/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:41.227459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8b786e1788045bfb01a4cb3e541d5f4, user_id=5b5ad45215294db0bde0e7bfc7512e02, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:41.228916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8385126-87c5-4702-b792-ce96b506aae2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:41.243110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 598/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:26:41 2022] PUT /identity/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4/users/5b5ad45215294db0bde0e7bfc7512e02/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:26:43.406077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0029-eb8e-403e-92ed-19cd69bb69e4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:43.406661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0029-eb8e-403e-92ed-19cd69bb69e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:43.406661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b0029-eb8e-403e-92ed-19cd69bb69e4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:43.408006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1193] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:26:43.414341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:43.415059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:43.415202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:43.436733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:26:43.438281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:43.457580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:43.525743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 599/1194] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:26:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:26:43.550351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0029-eb8e-403e-92ed-19cd69bb69e4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:43.555161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:43.678209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b0029-eb8e-403e-92ed-19cd69bb69e4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:43.721146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:43.721939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:43.722220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:43.722492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:43.733447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:26:43.740979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:26:43.742068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:26:43.749108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:26:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:26:43.750425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:43.859539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ab50e-f9e9-4edc-a3e7-69ee7118b609 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:43.894733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:43.895381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:43.895661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:43.895930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:43.896696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:43.896997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:43.898378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:43.906581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 600/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:26:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:26:43.912713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:43.992413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:44.034602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c3a72-f889-4dba-8cd4-277ccfd7245d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:44.070195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:44.071061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:44.071342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:44.071619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] PATH_INFO: `/v3/users/5b5ad45215294db0bde0e7bfc7512e02` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:44.072950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] RBAC: Authorizing `identity:get_user(user_id=5b5ad45215294db0bde0e7bfc7512e02)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:44.074433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:44.076071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Tue May 24 15:26:43 2022] GET /identity/v3/users/5b5ad45215294db0bde0e7bfc7512e02 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:44.082297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:44.095755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:44.185533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0515-8a36-4838-a9fd-0aada96557eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:44.221026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:44.221664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:44.221954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:44.222219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:44.291540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:44.292953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:44.304210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:44.394767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 601/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:26:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:26:44.401598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:44.516612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606db4b7-9d42-47a9-bd50-90b99909bba2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:44.546112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:44.552728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:44.553591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:44.553873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:44.554147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] PATH_INFO: `/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:44.555327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] RBAC: Authorizing `identity:get_project(project_id=b8b786e1788045bfb01a4cb3e541d5f4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:44.556826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d48336-a9e5-4fed-8f61-2d468a2502e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:44.558516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Tue May 24 15:26:44 2022] GET /identity/v3/projects/b8b786e1788045bfb01a4cb3e541d5f4 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:44.565062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:44.668696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cebeb19-14e7-4993-bc0e-8818c6c6c397 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:44.680879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:44.711263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:44.711910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:44.712211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:44.712481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:44.713448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=b8b786e1788045bfb01a4cb3e541d5f4, user.id=5b5ad45215294db0bde0e7bfc7512e02) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:26:44.713759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:44.715188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23cc1b-1fc7-49bc-be52-352a7b225d90 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:44.720936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 602/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:26:44 2022] GET /identity/v3/role_assignments?user.id=5b5ad45215294db0bde0e7bfc7512e02&scope.project.id=b8b786e1788045bfb01a4cb3e541d5f4&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:26:54.551980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:54.718756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:44.695036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb493-2215-491d-b290-7d794793b47c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:44.695665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb493-2215-491d-b290-7d794793b47c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:44.695665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfb493-2215-491d-b290-7d794793b47c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:44.700083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1201] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:27:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:27:44.704014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:44.704554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:44.704655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:44.727212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:27:44.731002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:27:44.824123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 603/1202] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:27:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:27:44.848012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfb493-2215-491d-b290-7d794793b47c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:44.980127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfb493-2215-491d-b290-7d794793b47c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:45.019940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:45.020785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:45.021043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:45.021340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:45.035212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:45.036714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:45.056769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:45.057934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:45.058979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:45.058979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:45.058979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:45.058979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:45.058979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:45.064181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:45.187070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2d84eb-e273-4d43-a7c0-328efea5a00e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:45.218340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:45.227822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:45.227822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:45.227822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:45.227822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:45.228424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:45.228424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:45.229075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55eb60-4c83-4d61-abcd-5743c298b9ee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:45.242460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 604/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:45.248716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:45.310249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:45.360580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436e0cf-8230-4675-9116-d7d508b1fd34 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:45.407435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:45.408269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:45.408610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:45.408928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:45.409633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:45.411130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aee1f5-2049-4836-975e-dbc4f0bd27c5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:45.432576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Tue May 24 15:27:45 2022] POST /identity/v3/users => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:27:47.967867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe210c0-6d92-436f-a028-7d8e06275ffe None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:47.968473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe210c0-6d92-436f-a028-7d8e06275ffe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:47.968473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe210c0-6d92-436f-a028-7d8e06275ffe None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:47.970083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 605/1206] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:27:47.975971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:47.976461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:47.976547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:47.995114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:27:47.996470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:27:48.074265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1207] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:27:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:27:48.098804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe210c0-6d92-436f-a028-7d8e06275ffe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:48.207307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe210c0-6d92-436f-a028-7d8e06275ffe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:48.243396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:48.244255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:48.244532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:48.244800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:48.257093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:27:48.266055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:48.267239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:48.273702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 606/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:27:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:27:48.274838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:48.319701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:48.388220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0970c9-c536-4f7e-8662-68e842649c9a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:48.426216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:48.426991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:48.427403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:48.427622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:48.428470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:48.428850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:48.430351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:48.440622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:27:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:48.447048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:48.503097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:48.560531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade1e3b-6a14-4a63-8c04-bb525d22d4b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:48.599632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:48.601207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:48.601207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:48.601207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:48.614497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:48.616350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:48.636812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:48.638011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:48.644140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:48.644140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:48.644140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:48.644140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:48.644140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 607/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:48.646127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:48.760754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:48.764116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7144a79-8f99-42fb-94c0-4031f3f156b2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:48.805856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:48.806556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:48.806837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:48.807109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:48.807837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:48.808223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:48.811733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:48.824791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:27:48.831057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:48.947684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b8c2-8dc5-49c7-9276-9875244e5e80 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:48.949671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:48.990811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:48.991683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:48.991964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:48.992314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:49.000193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:49.001809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:27:49.020247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:49.022127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:49.023112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 608/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:49.027694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:49.147532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ceb439-6589-4641-a47e-1d830beffc6c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:49.186120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:49.186846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:49.187148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:49.187463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:49.188377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:49.188715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:49.190287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:49.213995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:27:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:27:49.215338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:49.329990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73ccc8-28ce-417e-97d9-e10454a0408c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:49.367682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:49.368495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:49.368752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:49.369025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:49.403242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:49.447867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:49.450229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:49.567496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 609/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:27:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:27:49.573162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:49.595439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:49.694216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409637f7-3f2c-437e-bcca-9d3df4d74397 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:49.734675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:49.735525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:49.735858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:49.736264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:49.747125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:49.748645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:49.767965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:49.769013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:49.769958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:49.769958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:49.769958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:49.769958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:49.776353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:49.776494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:49.855463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:49.900212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd16d4-f38f-406a-9982-f6c7ffdcbbac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:49.940910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:49.941332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:49.941741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:49.941896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:49.942585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:49.942924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:49.944428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:49.963039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 610/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:49.964650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:50.047786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:50.085945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42498300-3359-40e7-a74d-cdca02c9489b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:50.126136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:50.126907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:50.127189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:50.127464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:50.135657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:50.137846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:27:50.153445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:50.154656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:50.155733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:50.162496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:27:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:50.164426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:50.273158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf304e-980d-4d17-95ce-9c54b1978b3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:50.297217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:50.312114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:50.312825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:50.313114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:50.313371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:50.314217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:50.314523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:50.316014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97a6e0e-6971-4487-8070-6d9db0df590c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:50.329068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 611/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:27:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:50.335762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:50.453113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fee9f-b17f-46f8-be4e-153c3735b6e4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:50.482597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:50.488742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c05137-7161-42d4-a336-414019215290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:50.489734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05137-7161-42d4-a336-414019215290 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:50.490015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05137-7161-42d4-a336-414019215290 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:50.490287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c05137-7161-42d4-a336-414019215290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:50.491121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c05137-7161-42d4-a336-414019215290 None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=3e5a44c30a0b430f90cabf9b3084ec28) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:50.491430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c05137-7161-42d4-a336-414019215290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:50.492922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c05137-7161-42d4-a336-414019215290 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:50.502151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:27:50 2022] GET /identity/v3/role_assignments?user.id=3e5a44c30a0b430f90cabf9b3084ec28&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:52.803118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f288d-b229-49dd-a360-d0874f262fb4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:52.803697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f288d-b229-49dd-a360-d0874f262fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:52.803697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f288d-b229-49dd-a360-d0874f262fb4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:52.805198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 612/1220] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:27:52.811138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:52.811658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:52.811758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:52.830366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:27:52.831661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:27:52.916190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1221] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:27:52.942242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f288d-b229-49dd-a360-d0874f262fb4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.069217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f288d-b229-49dd-a360-d0874f262fb4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:53.121453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:53.122250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:53.122529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:53.122799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:53.133322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:27:53.141010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:53.141993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:53.142821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 613/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:27:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:53.148636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.266674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba01d38-fedd-446b-bd53-3b3254ab5a53 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:53.292009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:53.304149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:53.304822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:53.305107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:53.305375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:53.306117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:53.306417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:53.307794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:53.315791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:27:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:53.321764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.366189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:53.426397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72711db1-7da1-4c73-9fc4-886cc32578ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:53.467091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:53.467911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:53.468263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:53.468562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:53.481235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:53.482828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:53.509706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:53.510658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:53.511610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:53.511610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:53.511610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:53.511610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:53.517381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 614/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:27:53.517519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.631927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274d403-d1ed-4d11-8efe-303fc9c258be None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:53.671979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:53.672685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:53.673068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:53.673440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:53.674238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:53.674627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:53.676156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:53.691671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:53.697734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.803182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:53.810789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e98578-2bf6-460e-ac32-3d64c0fe0f99 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:53.846891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:53.847794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:53.848132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:53.848424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:53.856856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:53.858525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:27:53.873851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:53.874977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:53.881551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 615/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:53.882292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:53.931799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:53.987639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83440cb2-c135-4727-a536-08672304fad4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.024995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.025757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.026061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.026335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.027424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:54.027823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.029550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.048567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:27:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:54.054966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:54.165760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af25477-19b5-4976-b8b8-933be85050ba None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.202670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.203553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.203879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.204206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.217384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.218914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:54.237427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:54.238557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:54.239644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:54.239644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:54.239644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:54.239644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:54.243606 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 616/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:54.244814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:54.354468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e8e9-0871-4a18-a385-38258394a78c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.356928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:54.392661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.393424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.393704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.393975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.394914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:54.395380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.397368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.410885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:54.417604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:54.428909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:54.540778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3a139-f87b-447e-bb0e-978f64068ff9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.581662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.582526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.582806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.583076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.590084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.591673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:27:54.609788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:54.610881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:54.611837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 617/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:27:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:54.617541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:54.728558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a9a3e-7f78-4b46-a8d3-11166cb29c87 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.766847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.767844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.768313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.768731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.769972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:54.770506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.773134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d058e3-2d13-4f9f-b588-41a6d420f37b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.791320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:27:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:27:54.796689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:54.871623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:54.914082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e22adb-866f-44c5-a8c8-7e11dcfd45b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:54.956018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:54.956954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:54.957312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:54.957590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/3e5a44c30a0b430f90cabf9b3084ec28/roles/28e70d7a30c941a897b58dba94f315a2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:54.975536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=3e5a44c30a0b430f90cabf9b3084ec28, role_id=28e70d7a30c941a897b58dba94f315a2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:54.977256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a113825-ca47-4095-af04-df47b16d2198 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:54.992821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 618/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:27:54 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/3e5a44c30a0b430f90cabf9b3084ec28/roles/28e70d7a30c941a897b58dba94f315a2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:27:57.273291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee1dd-f40e-42ec-b3a2-abe887d9897d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:57.273904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee1dd-f40e-42ec-b3a2-abe887d9897d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:57.273904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ee1dd-f40e-42ec-b3a2-abe887d9897d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:57.275209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1233] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:27:57.282477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:57.282945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:57.282945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:57.300431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:27:57.301628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:27:57.377164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 619/1234] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:27:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:27:57.401890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ee1dd-f40e-42ec-b3a2-abe887d9897d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:57.432622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:57.527830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ee1dd-f40e-42ec-b3a2-abe887d9897d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:57.567749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:57.568610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:57.568890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:57.569174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:57.581325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:27:57.590061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:57.591221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:57.592230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Tue May 24 15:27:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:27:57.597484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:57.652861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:57.719323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c130864-1b93-47c4-8e21-d4dcddab8aa4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:57.755598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:57.756305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:57.756589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:57.756863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:57.757611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:57.757922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:57.759524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:57.768958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 620/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Tue May 24 15:27:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:57.774919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:57.879961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:57.899351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57c71e-4122-4d89-8383-36a36d476b0d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:57.938794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:57.939691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:57.940073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:57.940470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:57.951966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:57.953481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:57.971672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:57.972800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:57.974095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:57.974095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:57.974095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:57.974095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:57.974095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:57.978976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:58.088049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe511-1520-4048-9aca-35a511f2f8ca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:58.099542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:58.130205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:58.130863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:58.131138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:58.131412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:58.132162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:58.132464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:58.133840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:58.148692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 621/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:58.154619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:58.263947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea13a-9ec8-44d8-a5f5-0ac59b4bbc4b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:58.301309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:58.302124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:58.302395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:58.302669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:58.305601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:58.311528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:58.313200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:27:58.329253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:58.330592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:27:58.333459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:58.336772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:58.453430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3baa4d-4519-465e-9772-9170d714104c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:58.493068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:58.493736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:58.494011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:58.494279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:58.495107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:58.495444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:58.497081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:58.517903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 622/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:27:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:58.523710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:58.635350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af688b7-4e01-4d0a-b982-9c157a7a6098 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:58.671704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:58.672493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:58.672755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:58.673028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:58.740286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:58.747448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:58.748925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:58.857562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:27:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:27:58.864174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:58.895801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:58.981243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53713793-40de-4911-afe2-02bd84f5e951 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:59.020568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:59.021305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:59.021609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:59.021886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:59.033389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:59.035299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:27:59.056760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:59.058155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:59.059320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:27:59.059320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:27:59.059320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:27:59.059320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:59.059320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 623/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:27:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:59.064993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:59.169476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:59.176822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc44021-d1d4-4925-868e-f2022755e5b8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:59.217049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:59.217758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:59.218050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:59.218354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:59.219102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:59.219438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:59.221198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:59.235654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:27:59.241839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:59.332012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:27:59.354407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258da8c4-fe1d-4ba5-a293-2b6884cc6776 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:59.389546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:59.390345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:59.390621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:59.390893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:59.398589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:59.400424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:27:59.414995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:27:59.416384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:27:59.417647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 624/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:27:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:27:59.422962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:59.544125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512513e-e116-4508-b809-9b8f4f075bc7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:59.585174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:59.585777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:59.586054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:59.586394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:59.587177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:59.587530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:59.589276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4834ad2-4ed1-40e1-b5b3-d6e1015f3718 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:59.602625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:27:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:27:59.609248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:27:59.723100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b3c190-c7ee-462a-955a-5bea66917ee3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:27:59.761338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:27:59.762038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:27:59.762309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:27:59.762576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:27:59.763453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] RBAC: Adding query filter params (role.id=28e70d7a30c941a897b58dba94f315a2, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=3e5a44c30a0b430f90cabf9b3084ec28) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:27:59.763760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:27:59.765234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea3a1-6681-40c1-a4c5-267e00f7a19c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:27:59.771307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 625/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:27:59 2022] GET /identity/v3/role_assignments?user.id=3e5a44c30a0b430f90cabf9b3084ec28&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=28e70d7a30c941a897b58dba94f315a2 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:02.153324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef32b-6c8b-4b3c-aa56-355a303c2f8b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:02.153883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef32b-6c8b-4b3c-aa56-355a303c2f8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:02.153883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ef32b-6c8b-4b3c-aa56-355a303c2f8b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:02.155331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1247] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:02.161397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:02.163522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:02.163522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:02.168819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:02.184166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:02.185603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:02.261066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 626/1248] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:28:02.287777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef32b-6c8b-4b3c-aa56-355a303c2f8b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:02.308890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:02.402745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ef32b-6c8b-4b3c-aa56-355a303c2f8b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:02.447612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:02.448575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:02.448978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:02.449279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:02.464652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:28:02.473252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:02.474582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:02.476095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:02.481509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:02.607732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3103c217-3515-4c60-943f-59b22fa32920 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:02.640438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:02.645848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:02.646725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:02.647263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:02.647556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:02.648394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:02.648700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:02.650612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:02.659883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 627/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:28:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:02.666850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:02.758182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:02.783542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b329be-382a-408f-a51a-41306381fad4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:02.819357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:02.820230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:02.820536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:02.820870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:02.832268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:02.833715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:02.851882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:02.853189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:02.860059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:02.860059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:02.860059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:02.860059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:02.860059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:02.860421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:02.969834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19de5-77be-4942-9c60-f10639a7d7b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:03.006524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:03.007220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:03.007499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:03.007774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:03.009936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:03.010201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:03.012085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:03.027387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 628/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:03.036384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:03.160041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9969b8f-bea9-44ce-82f7-0ecbf05f0d65 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:03.186502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:03.195044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:03.195860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:03.196242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:03.196518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:03.204512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:03.206055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:28:03.220816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:03.221976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:03.223120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:03.227017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:03.228181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:03.294681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:03.348042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313b33c-80ed-4351-8e88-e66b59d7686c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:03.384992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:03.385653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:03.386095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:03.386354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:03.387190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:03.387490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:03.388951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:03.411910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 629/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:28:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:03.418175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:03.538851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc310e-6aae-4b49-97f1-30620e6deeab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:03.581060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:03.581716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:03.582080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:03.582355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:03.651329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:03.665356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:03.667163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:03.777192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:28:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:28:03.783181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:03.886090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaa801-14d1-4534-9e8e-26991bb0b209 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:03.899985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:03.923494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:03.924353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:03.924630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:03.924903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:03.935582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:03.937063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:03.954415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:03.955626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:03.956840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:03.956840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:03.956840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:03.956840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:03.956840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 630/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:03.961804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:04.058070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:04.071297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170e468-dc21-4ba2-84d5-f7317ea3a938 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:04.109399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:04.110245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:04.110589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:04.110992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:04.111802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:04.112239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:04.113870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:04.127198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:04.133501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:04.241686 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846ab8f-1092-4508-ab8a-06ff6b250616 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:04.282278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:04.283165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:04.283447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:04.283742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:04.291618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:04.293424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:28:04.308680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:04.309916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:04.311105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:04.312427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 631/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:28:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:28:04.316303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:04.431835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563f16a-a91c-4a2d-9f03-f39da6d635af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:04.476217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:04.476965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:04.477256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:04.477535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:04.478443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:04.478778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:04.480456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65563fa-1fda-41d7-9282-910cd79748e6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:04.500204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:28:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:04.501448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:04.528515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:04.610612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebc485-6c03-4fa2-a9e1-307c7504620b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:04.648412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:04.649083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:04.649363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:04.649637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:04.650480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=3e5a44c30a0b430f90cabf9b3084ec28) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:04.650792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:04.652455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1156786e-ccb1-4092-895f-754ccea3a54d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:04.659218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 632/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:28:04 2022] GET /identity/v3/role_assignments?user.id=3e5a44c30a0b430f90cabf9b3084ec28&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:06.867090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676b6-6b85-4e42-b16d-2afcde6c40e6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:06.867925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676b6-6b85-4e42-b16d-2afcde6c40e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:06.867925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676b6-6b85-4e42-b16d-2afcde6c40e6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:06.870328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1261] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:06.878312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19554-c076-4bb0-985e-df310387791e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:06.878776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19554-c076-4bb0-985e-df310387791e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:06.878877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed19554-c076-4bb0-985e-df310387791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:06.897504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7ed19554-c076-4bb0-985e-df310387791e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:06.898683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ed19554-c076-4bb0-985e-df310387791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:06.968193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 633/1262] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:28:06.991705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1676b6-6b85-4e42-b16d-2afcde6c40e6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.005139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:07.105278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1676b6-6b85-4e42-b16d-2afcde6c40e6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:07.140239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:07.141017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:07.141292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:07.141562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:07.152527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:28:07.160220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:07.161303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:07.162364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:28:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:07.169864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed19554-c076-4bb0-985e-df310387791e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.189467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:07.273172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed19554-c076-4bb0-985e-df310387791e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:07.308615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:07.309269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:07.309548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:07.309820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:07.310558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:07.310863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:07.312269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:07.320440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 634/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:28:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:07.328111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.407346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:07.447520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb3adf-60da-401d-86f7-e2c7bb06e7d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:07.484536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:07.485432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:07.485604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:07.485936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:07.498356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:07.499840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:07.520883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:07.521919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:07.523008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:07.523008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:07.523008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:07.523008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:07.523008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:07.528603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.620021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:07.643867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a69b3-3112-44e3-891f-1497e85de485 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:07.681275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:07.681935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:07.682212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:07.682528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:07.683253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:07.683551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:07.684978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:07.697167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 635/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:07.703096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.807886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730e6c0-2072-4fff-9c40-570ddd3b163b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:07.833830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:07.850214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:07.851014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:07.851288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:07.851559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:07.858972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:07.860450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:28:07.874691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:07.876379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:07.877639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:07.882265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:07.992776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9624427-c267-414e-8ce8-cb5c0bf0d3d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.033137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.033786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.034064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.034331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.035163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:08.035470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.036931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.046476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:08.059986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 636/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:28:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:28:08.065780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:08.167793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b713a-8a2e-4550-aad5-fad3f3365727 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.202873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.203647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.203920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.204220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.215027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.216504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:08.234625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:08.235608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:08.236720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:08.236720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:08.236720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:08.236720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:08.240190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:08.241360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:08.347386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773fc67-c69a-4386-9e37-f77af8bbc0f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.383704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.384376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.384649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.384916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.385700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:08.385995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.387358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.398924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 637/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:08.408062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:08.446597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:08.519742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c459f88-fac1-4cfc-a82e-855057dc23dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.556423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.557215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.557537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.557837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.564602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.566068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:28:08.580179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:08.581198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:08.582962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:28:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:08.590007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:08.654992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:08.701137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ee99a-8581-4351-862a-d5f2d27595db None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.734772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.735410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.735683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.735953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.736764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:08.737076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.738500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb24de-dd17-482e-8b47-8a8b5ceb1061 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.750511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 638/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:28:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:08.756738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:08.855715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:08.869974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d1f53-76cd-4518-af33-17b93c53b3fa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:08.912048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:08.913191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:08.913466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:08.913733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] PATH_INFO: `/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/3e5a44c30a0b430f90cabf9b3084ec28/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:08.916097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c13cc48481ea47fca3a3ef6370ade58e, user_id=3e5a44c30a0b430f90cabf9b3084ec28, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:08.917614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425f306-2a4a-43d7-a2d3-515cca5003f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:08.934197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Tue May 24 15:28:08 2022] PUT /identity/v3/projects/c13cc48481ea47fca3a3ef6370ade58e/users/3e5a44c30a0b430f90cabf9b3084ec28/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:28:11.084616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e091c2-ef69-4816-ae17-38f531c99fc2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.085174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e091c2-ef69-4816-ae17-38f531c99fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.085174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e091c2-ef69-4816-ae17-38f531c99fc2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.086542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 639/1274] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:11.093089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.093804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.093948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.115600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:11.116806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:11.187815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1275] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:28:11.215799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e091c2-ef69-4816-ae17-38f531c99fc2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:11.322714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e091c2-ef69-4816-ae17-38f531c99fc2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:11.341898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:11.369217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:11.370398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.370731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.370998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.381987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:28:11.389958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:11.390991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:11.391855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 640/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Tue May 24 15:28:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:11.397305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:11.492189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:11.512760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c8086-afa1-40ce-b932-79ec9767f2a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:11.547815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:11.548562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.548833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.549101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.549834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:11.550133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:11.551506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:11.559873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Tue May 24 15:28:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:11.565880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:11.676934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4eb0c-33c1-4dea-8172-f27194e1bc10 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:11.711671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:11.712490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.712766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.713039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.724005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:11.725477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:11.742801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:11.743879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:11.750251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:11.750251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:11.750251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:11.750251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:11.750251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 641/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:11.752472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:11.857120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a50d3-0cba-43c2-af12-2141417f8fb4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:11.897308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:11.898336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:11.898749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:11.899150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:11.900335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:11.900773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:11.903085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:11.921321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:11.930297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:11.969775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:12.041251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153d7ce-9d1b-45d9-ab62-b6d16a5eebc5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:12.076437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:12.077217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:12.077496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:12.077762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:12.085467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:12.086906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:28:12.101266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:12.102239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:12.103180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 642/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:12.108512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:12.112560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:12.223970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d7d1c-157d-421a-940a-5835837ab307 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:12.257584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:12.258239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:12.258517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:12.258780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:12.259606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:12.259902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:12.261486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:12.278272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Tue May 24 15:28:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:12.284064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:12.379487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:12.389330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd2319d-7301-4cd9-8ac2-0b6cfdec9ffc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:12.433376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:12.434140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:12.434474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:12.434749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:12.502068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:12.503451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:12.606128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:12.617413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 643/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Tue May 24 15:28:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:28:12.623840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:12.686773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:12.726051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f428992-d6a4-4cb0-a707-d71fe7497738 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:12.761474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:12.762199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:12.762472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:12.762736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:12.773120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:12.774565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:28:12.792340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:12.793340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:12.794662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:28:12.794662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:28:12.794662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:28:12.794662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:12.794662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:28:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:28:12.799380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:12.899385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65fc35-7263-4d94-bc74-f661101711e7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:12.947380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:12.948073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:12.950701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:12.950995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:12.951731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:12.952053 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:12.953442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:12.966106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 644/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Tue May 24 15:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:12.975431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:13.107402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d133f-8e99-4c91-a013-fd0bd47be7f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:13.122086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:13.144906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:13.145714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:13.145995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:13.146262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:13.153931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:13.155384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:28:13.169842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:13.170815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: ERROR keystone.server.flask.application May 24 15:28:13.173587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Tue May 24 15:28:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:13.180040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:13.237156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:13.292430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309156c0-691d-46ff-9015-6c4059d5c610 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:13.331891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:13.332566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:13.332847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:13.333114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:13.333898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:13.334210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:13.335656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca265ba-1114-4ba9-b711-1d45e0b6acb3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:13.349681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 645/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Tue May 24 15:28:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:28:13.359184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:13.505822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc12aa-c019-4a13-b401-ba841cde9e8d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:13.541106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:13.541748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:13.542031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:13.542289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:13.543129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] RBAC: Adding query filter params (role.id=1d5097d60ffe4e02b13f5c0cc5227bc6, scope.project.id=c13cc48481ea47fca3a3ef6370ade58e, user.id=3e5a44c30a0b430f90cabf9b3084ec28) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:13.543432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:13.544898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9808f-fe4b-48bc-99ea-ae342f86b181 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:13.553964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Tue May 24 15:28:13 2022] GET /identity/v3/role_assignments?user.id=3e5a44c30a0b430f90cabf9b3084ec28&scope.project.id=c13cc48481ea47fca3a3ef6370ade58e&role.id=1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:15.816473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5c6a-6d10-478c-9727-a3c6ba120731 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:15.817311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5c6a-6d10-478c-9727-a3c6ba120731 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:15.817311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5c6a-6d10-478c-9727-a3c6ba120731 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:15.819463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 646/1288] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:15.828218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:15.829685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:15.829685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:15.849199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:15.850400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:15.884529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:15.927127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1289] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:28:15.949609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5c6a-6d10-478c-9727-a3c6ba120731 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:16.065881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5c6a-6d10-478c-9727-a3c6ba120731 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:16.076951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:16.106401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:16.107196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:16.107477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:16.107747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:16.108427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:16.109797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:16.121987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:16.123173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:16.124444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 647/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:28:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:16.131282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:16.234870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eaafe1-eb28-46e2-bb9b-b8546241dd75 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:16.270628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:16.271286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:16.271567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:16.271843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:16.272581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:16.272885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:16.274282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bca8d7-a9c6-46f1-a2a7-e492618d0ff6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:16.279369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Tue May 24 15:28:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:16.285395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:16.393861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8dd18-36fa-4509-b9bd-3bb1c631703e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:16.436111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:16.437007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:16.437388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:16.437783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:16.438841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:16.439286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:16.440800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e2f2c-a3e8-4317-bfdf-3a9281350074 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:16.445804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 648/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Tue May 24 15:28:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:18.603424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd1fe-55d3-451c-a1e7-be13321277f8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:18.604051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd1fe-55d3-451c-a1e7-be13321277f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:18.604051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedd1fe-55d3-451c-a1e7-be13321277f8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:18.605458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1293] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:18.611755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431136b0-ada2-4e59-9b7a-7bfee55e14a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:18.612568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431136b0-ada2-4e59-9b7a-7bfee55e14a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:18.612723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431136b0-ada2-4e59-9b7a-7bfee55e14a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:18.638819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-431136b0-ada2-4e59-9b7a-7bfee55e14a2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:18.640011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-431136b0-ada2-4e59-9b7a-7bfee55e14a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:18.671332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:18.712619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 649/1294] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:28:18.735411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd1fe-55d3-451c-a1e7-be13321277f8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:18.771881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:18.845196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedd1fe-55d3-451c-a1e7-be13321277f8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:18.879870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:18.880478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:18.880755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:18.881025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:18.881731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:18.883095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06139d7-e585-4347-8520-cb8f167c1d2c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:18.900479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Tue May 24 15:28:18 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:28:21.062585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d53b5-77b2-444b-9e60-549fb3499e40 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.063436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d53b5-77b2-444b-9e60-549fb3499e40 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.063436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d53b5-77b2-444b-9e60-549fb3499e40 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.065692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 650/1296] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:21.073885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.074320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.074407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.092982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:21.094136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:21.170414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1297] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:28:21.200927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d53b5-77b2-444b-9e60-549fb3499e40 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:21.309950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d53b5-77b2-444b-9e60-549fb3499e40 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:21.349507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:21.350219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.350494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.350765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.351411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:21.352813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:21.365110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:21.366206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:21.367750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 651/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:28:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:21.373985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:21.392553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:21.483916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65314aa-dd94-45b7-8418-a3a9eae5c0fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:21.520369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:21.521028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.521306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.521575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.522276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:21.522581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:21.523950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:21.529524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Tue May 24 15:28:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:21.535644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:21.571683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:21.642059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896d1ee-e2ac-44ea-af27-abeccea3b456 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:21.686207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:21.686863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.687140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.687409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.688441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] RBAC: Adding query filter params (interface=public, service_id=ee96177fdb144a2e998ede6b7c3ff10c, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:21.688536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:21.689954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827c8c9-485a-46e1-a5df-13b2e7c1b5e4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:21.696501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 652/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Tue May 24 15:28:21 2022] GET /identity/v3/endpoints?service_id=ee96177fdb144a2e998ede6b7c3ff10c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:21.703585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:21.796885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:21.816153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d74a7e-3d24-4fb0-a723-cdc571a536e0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:21.851102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:21.851760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:21.852075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:21.852341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:21.853013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:21.854469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9f436-f5a1-4cd2-8411-19f9154e6d3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:21.861359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Tue May 24 15:28:21 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:28:24.017512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c878772-9b26-4dca-b1df-a2742a898136 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:24.018062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c878772-9b26-4dca-b1df-a2742a898136 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:24.018062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c878772-9b26-4dca-b1df-a2742a898136 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:24.019409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 653/1302] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:28:24.026344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:24.026837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:24.026837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:24.046295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:28:24.047473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:24.129027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1303] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:28:24 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:28:24.152281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c878772-9b26-4dca-b1df-a2742a898136 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:24.268013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:24.276793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c878772-9b26-4dca-b1df-a2742a898136 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:24.314935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:24.315715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:24.315992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:24.316292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:24.316939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:24.318303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:28:24.330398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:28:24.331531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application May 24 15:28:24.332582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 654/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Tue May 24 15:28:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:28:24.338729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:24.417837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:24.441950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4520bc0-b88d-453f-982f-135e8f0de74a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:24.480943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:24.481597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:24.481873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:24.482143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:24.482842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:28:24.483143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:24.484572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab5e82-4a52-4a43-8c31-668b27d31aed None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:24.489908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Tue May 24 15:28:24 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:28:24.496043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:24.606214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5641c-b0f7-4662-9a86-ae9f8ae801d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:24.642641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:24.643274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:24.643551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:24.643828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:24.644594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:24.645963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb6407-0377-4256-b9ad-de499b6cc031 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:24.668775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 655/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Tue May 24 15:28:24 2022] POST /identity/v3/endpoints => generated 331 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:28:34.486718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:34.663492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:28.189235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fc4b6-373d-43b4-99d1-022abd4bc595 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:28.189815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fc4b6-373d-43b4-99d1-022abd4bc595 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:28.189815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5fc4b6-373d-43b4-99d1-022abd4bc595 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:28.196493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677766-95b6-4ef7-bf7b-3af2f8b3b2b7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:28.197030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677766-95b6-4ef7-bf7b-3af2f8b3b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:28.197192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a677766-95b6-4ef7-bf7b-3af2f8b3b2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:28.200687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1307] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:28.288697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8a677766-95b6-4ef7-bf7b-3af2f8b3b2b7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:28.289914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8a677766-95b6-4ef7-bf7b-3af2f8b3b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:28.458801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 656/1308] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:29:28.524437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:28.524778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:28.525024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:28.526834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1309] 10.222.0.55 () {54 vars in 964 bytes} [Tue May 24 15:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:28.539352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc2fd8-2ad2-4276-9d6f-78369141e329 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:28.539642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc2fd8-2ad2-4276-9d6f-78369141e329 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:28.540519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc2fd8-2ad2-4276-9d6f-78369141e329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:28.696584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3fbc2fd8-2ad2-4276-9d6f-78369141e329 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:28.697989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3fbc2fd8-2ad2-4276-9d6f-78369141e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:28.739011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:28.855292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 657/1310] 10.222.0.55 () {58 vars in 1061 bytes} [Tue May 24 15:29:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:28.865684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:28.870390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:29.032643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6e748-357b-4f7f-8565-d40857ae349a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:29.124412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:29.125479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:29.126019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:29.126489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:29.128998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:29.194295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:29.253155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:29.255001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea1ce8-99a7-40c4-9ff5-60a6e2d5d8d0 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:29.278757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:31.773534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:31.774044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:31.774154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:31.776069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 658/1312] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:31.781101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850033f9-c702-49b3-957f-59985e5f16c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:31.781585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850033f9-c702-49b3-957f-59985e5f16c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:31.781863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850033f9-c702-49b3-957f-59985e5f16c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:31.857772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-850033f9-c702-49b3-957f-59985e5f16c3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:31.858985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-850033f9-c702-49b3-957f-59985e5f16c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:31.959359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1313] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:31.996972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:31.998521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:32.107712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f651059-a128-4c05-b8d1-983243bdee51 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:32.180753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:32.183612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:32.184345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:32.184562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:32.184867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:32.186929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:32.300114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:32.301537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ca626-4923-426e-a48e-0ad3ccc96ff3 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:32.320150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 659/1314] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:34.842554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bae27-d349-44cb-81d8-455754074be4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:34.843130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bae27-d349-44cb-81d8-455754074be4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:34.843130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bae27-d349-44cb-81d8-455754074be4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:34.844563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1315] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:34.850868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73427b-36f2-4320-b298-2ad95ff7cf4c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:34.851593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73427b-36f2-4320-b298-2ad95ff7cf4c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:34.851774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba73427b-36f2-4320-b298-2ad95ff7cf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:34.927755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ba73427b-36f2-4320-b298-2ad95ff7cf4c None None] MFA Rules not processed for user `d4c637fab90949129383c9f979fffa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:34.929037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba73427b-36f2-4320-b298-2ad95ff7cf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:34.983329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:35.028528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 660/1316] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:35.084692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:35.084965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:35.085361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:35.087232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1317] 10.222.0.55 () {54 vars in 964 bytes} [Tue May 24 15:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:35.095746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3526e4-8d09-4eec-b5f2-e988364767c7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:35.096050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3526e4-8d09-4eec-b5f2-e988364767c7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:35.096337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3526e4-8d09-4eec-b5f2-e988364767c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:35.175681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4e3526e4-8d09-4eec-b5f2-e988364767c7 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:35.176938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e3526e4-8d09-4eec-b5f2-e988364767c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:35.259479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 661/1318] 10.222.0.55 () {58 vars in 1061 bytes} [Tue May 24 15:29:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:29:35.273129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:35.275759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:35.286398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:35.389950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3a59d-9c38-4a34-9ce9-735caddba204 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:35.469059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:35.469693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:35.469968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:35.470237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:35.472170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:35.535009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:35.586143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:35.587689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe10d80-fa7b-4aa4-a63e-00241cad5f9d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:35.606872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1319] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:38.755645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:38.756220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:38.756220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:38.757628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 662/1320] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:38.764099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119e18-3a2b-4a08-bb3e-5c3b3d8c9415 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:38.764825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119e18-3a2b-4a08-bb3e-5c3b3d8c9415 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:38.765007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b119e18-3a2b-4a08-bb3e-5c3b3d8c9415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:38.844671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5b119e18-3a2b-4a08-bb3e-5c3b3d8c9415 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:38.846553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5b119e18-3a2b-4a08-bb3e-5c3b3d8c9415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:38.953713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1321] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:38.991774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:38.993353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:39.047822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:39.105231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040f3f7-280d-4ef2-b478-723c33493716 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:39.179794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:39.180466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:39.180737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:39.181006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:39.182715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:39.289016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:39.305996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:39.308271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d869281-795f-4c58-8b1a-f334cbf5d1e6 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:39.329585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 663/1322] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:42.243725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:42.244323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:42.244440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:42.245784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1323] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:42.251195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b141-401d-4ee0-9aaf-255f9415846b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:42.251599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b141-401d-4ee0-9aaf-255f9415846b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:42.251775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b141-401d-4ee0-9aaf-255f9415846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:42.317825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f791b141-401d-4ee0-9aaf-255f9415846b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:42.319037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f791b141-401d-4ee0-9aaf-255f9415846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:42.425234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 664/1324] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:42.461324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:42.463081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:42.485787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:42.572747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8db56-d037-4fd7-8b24-39659d1d7799 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:42.655311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:42.655968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:42.656275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:42.656545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:42.658214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:42.735372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:42.776564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:42.778022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3914713-b2d8-4373-b55e-6751282c6e9b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:42.797477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1325] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:45.012938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:45.013474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:45.013474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:45.014916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 665/1326] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:45.020434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bc8c7-3d34-481b-93b6-945b5e8240dc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:45.020851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bc8c7-3d34-481b-93b6-945b5e8240dc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:45.020976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240bc8c7-3d34-481b-93b6-945b5e8240dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:45.084281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-240bc8c7-3d34-481b-93b6-945b5e8240dc None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:45.085498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-240bc8c7-3d34-481b-93b6-945b5e8240dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:45.188348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1327] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:29:45.223351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:45.224927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:45.289028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:45.332834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d8b2f-5b51-481c-ae1c-2131825658c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:45.407615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:45.408293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:45.408542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:45.408808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:45.410778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:45.526889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:45.529299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:45.530733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7fdeb-fabb-4484-80e3-17d5c56bfce1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:45.551427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 666/1328] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:47.778875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:47.779458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:47.779458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:47.780838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1329] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:47.786535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed1f9-0c32-4e58-88c9-4351ee6f00c7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:47.786986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed1f9-0c32-4e58-88c9-4351ee6f00c7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:47.787090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ed1f9-0c32-4e58-88c9-4351ee6f00c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:47.853830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eb6ed1f9-0c32-4e58-88c9-4351ee6f00c7 None None] MFA Rules not processed for user `d4c637fab90949129383c9f979fffa9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:47.855023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb6ed1f9-0c32-4e58-88c9-4351ee6f00c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:47.959258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 667/1330] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:47.996674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:47.998214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:48.032177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:48.109151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076a56d0-f05c-4689-86ab-be6ed6f41c45 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:48.185981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:48.186610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:48.186899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:48.187159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:48.188896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:48.271460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:48.304277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:48.305740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666ea13-6156-4dcc-b531-b5b817ffa393 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:48.324271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1331] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:50.774811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:50.775596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:50.776094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:50.777522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 668/1332] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:50.783252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11cef-a4b9-4d2e-a533-00bfabcf7fed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:50.783718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11cef-a4b9-4d2e-a533-00bfabcf7fed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:50.783813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a11cef-a4b9-4d2e-a533-00bfabcf7fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:50.853171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a6a11cef-a4b9-4d2e-a533-00bfabcf7fed None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:50.854438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a6a11cef-a4b9-4d2e-a533-00bfabcf7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:50.965114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1333] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:29:51.003318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:51.005906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:51.078478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:51.118151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15268174-b827-46b5-92d2-1882d1040bb2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:51.200579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:51.201243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:51.201522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:51.201794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:51.203467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:51.322238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:51.323775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8490a56-a69d-46a8-8e24-a7120d159833 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:51.334278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:51.343220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 669/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:53.571054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:53.571614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:53.571614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:53.572986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1335] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:53.579522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa55f-20fb-4e51-86f5-b9ccc1577fd8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:53.580295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa55f-20fb-4e51-86f5-b9ccc1577fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:53.580457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa55f-20fb-4e51-86f5-b9ccc1577fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:53.650918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-340fa55f-20fb-4e51-86f5-b9ccc1577fd8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:53.652148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-340fa55f-20fb-4e51-86f5-b9ccc1577fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:53.750493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 670/1336] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:53.792933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:53.794509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:53.837969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:53.911552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958d1d8-edae-4be2-b830-f3c6bad39383 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:53.991291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:53.991936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:53.992240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:53.992510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:53.994165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:54.085323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:54.113936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:54.115302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d929966-5e51-4c02-ae90-f9e847737160 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:54.134766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:29:56.877022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:56.877917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:56.877917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:56.880049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 671/1338] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:29:56.885530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0761f-cb12-4975-9b26-416a65b8dba1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:56.885968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0761f-cb12-4975-9b26-416a65b8dba1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:56.886079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0761f-cb12-4975-9b26-416a65b8dba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:56.956691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e7d0761f-cb12-4975-9b26-416a65b8dba1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:29:56.957928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7d0761f-cb12-4975-9b26-416a65b8dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:57.064382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1339] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:29:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:29:57.100583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:29:57.103191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:57.187268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:29:57.216387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e6329-fdfd-4f92-850b-f6d95ddf1c60 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:29:57.290550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:29:57.291232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:29:57.291501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:29:57.291765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:29:57.293425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:29:57.406550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:29:57.407981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988955-076c-4947-8366-9bfd63685e29 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:29:57.428297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 672/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:29:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:01.629043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:01.629635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:01.629635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:01.630893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1341] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:01.636304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f1ed7-0c94-4d45-b6d5-8fc2051a43d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:01.636762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f1ed7-0c94-4d45-b6d5-8fc2051a43d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:01.636862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f1ed7-0c94-4d45-b6d5-8fc2051a43d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:01.643561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:01.718237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a9f1ed7-0c94-4d45-b6d5-8fc2051a43d9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:01.719441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a9f1ed7-0c94-4d45-b6d5-8fc2051a43d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:01.829169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 673/1342] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:01.867707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:01.870192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:01.924667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:01.971123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d70f1-48bf-4100-ad78-f5518a8d8438 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:01.974881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:01.975535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:01.975811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:01.976112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:01.985611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:01.987085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab887d3-c2a8-4c36-95c7-4cf8b25f7251 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:01.997120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:04.901577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7e71-a813-4e4b-9d75-71701f9d47fc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:04.902131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7e71-a813-4e4b-9d75-71701f9d47fc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:04.902131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed7e71-a813-4e4b-9d75-71701f9d47fc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:04.903390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 674/1344] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:04.908793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515060-68d5-4338-931d-adcb296bd57b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:04.909246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515060-68d5-4338-931d-adcb296bd57b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:04.909368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e515060-68d5-4338-931d-adcb296bd57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:04.938251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e515060-68d5-4338-931d-adcb296bd57b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:04.939442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e515060-68d5-4338-931d-adcb296bd57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:05.018832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1345] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:05.055363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed7e71-a813-4e4b-9d75-71701f9d47fc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:05.067954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed7e71-a813-4e4b-9d75-71701f9d47fc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:05.072096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:05.072847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:05.073191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:05.073531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:05.082996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:05.084627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bc11c-7351-483e-99b6-4767cf1acb5f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:05.094550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 675/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:09.435089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e1aa-b0a3-4657-b5b2-852a614c7f2d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:09.435588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e1aa-b0a3-4657-b5b2-852a614c7f2d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:09.435698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4e1aa-b0a3-4657-b5b2-852a614c7f2d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:09.437234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1347] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:09.442755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65615acb-0386-46a6-ab1d-3af65cfdff40 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:09.443181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65615acb-0386-46a6-ab1d-3af65cfdff40 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:09.443271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65615acb-0386-46a6-ab1d-3af65cfdff40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:09.461208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-65615acb-0386-46a6-ab1d-3af65cfdff40 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:09.462402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65615acb-0386-46a6-ab1d-3af65cfdff40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:09.470460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 676/1348] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:09.508581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4e1aa-b0a3-4657-b5b2-852a614c7f2d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:09.520069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4e1aa-b0a3-4657-b5b2-852a614c7f2d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:09.523992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:09.524724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:09.525008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:09.525283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:09.534672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:09.536487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedd752-927d-460c-9bd0-4ed89de1fb59 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:09.545927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:15.049929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffce6d-5588-41b4-b616-82b0d336db0d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:15.050808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffce6d-5588-41b4-b616-82b0d336db0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:15.051250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffce6d-5588-41b4-b616-82b0d336db0d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:15.053921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 677/1350] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:30:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:15.063947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c7a3-5584-441e-8c67-4f9bfba981bf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:15.064735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c7a3-5584-441e-8c67-4f9bfba981bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:15.065156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c7a3-5584-441e-8c67-4f9bfba981bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:15.172466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-78a1c7a3-5584-441e-8c67-4f9bfba981bf None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:15.174729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78a1c7a3-5584-441e-8c67-4f9bfba981bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:15.219421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:15.300916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:30:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:19.455851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=72 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:19.859320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82045c2-bea8-496c-ba8e-1e0748bf5dae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:19.859842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82045c2-bea8-496c-ba8e-1e0748bf5dae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:19.860352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82045c2-bea8-496c-ba8e-1e0748bf5dae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:19.861683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 678/1352] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:19.866864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fd7d-327a-42cb-a380-7a086b049bfa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:19.867510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fd7d-327a-42cb-a380-7a086b049bfa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:19.867861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1fd7d-327a-42cb-a380-7a086b049bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:19.889603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-32f1fd7d-327a-42cb-a380-7a086b049bfa None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:19.890841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-32f1fd7d-327a-42cb-a380-7a086b049bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:19.902318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:30:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:21.398009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546725d-b790-421a-aafe-94a82072efe6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:21.398577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546725d-b790-421a-aafe-94a82072efe6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:21.398577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546725d-b790-421a-aafe-94a82072efe6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:21.399902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 679/1354] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:30:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:21.406322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bfd90-b2b7-4fd2-894d-506a1e78b1d1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:21.406771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bfd90-b2b7-4fd2-894d-506a1e78b1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:21.406872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bfd90-b2b7-4fd2-894d-506a1e78b1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:21.425877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-838bfd90-b2b7-4fd2-894d-506a1e78b1d1 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:21.427056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-838bfd90-b2b7-4fd2-894d-506a1e78b1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:21.436840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:30:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:25.190985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd11ef-2310-4c5a-9364-3c1dc6f84930 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:25.191920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd11ef-2310-4c5a-9364-3c1dc6f84930 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:25.192235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd11ef-2310-4c5a-9364-3c1dc6f84930 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:25.193692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 680/1356] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:25.199165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7e64-ee2e-4369-ab41-fe5fef1ef40f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:25.199595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7e64-ee2e-4369-ab41-fe5fef1ef40f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:25.199770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7e64-ee2e-4369-ab41-fe5fef1ef40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:25.218527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1ed7e64-ee2e-4369-ab41-fe5fef1ef40f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:25.219715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1ed7e64-ee2e-4369-ab41-fe5fef1ef40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:25.228633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1357] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:25.283561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ca71-e35a-441e-9402-7c6ce94d0d76 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:25.283957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ca71-e35a-441e-9402-7c6ce94d0d76 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:25.284354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066ca71-e35a-441e-9402-7c6ce94d0d76 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:25.286148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 681/1358] 10.222.0.55 () {54 vars in 961 bytes} [Tue May 24 15:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:25.294181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2b86-6abd-49c7-8347-dcb438d90382 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:25.294465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2b86-6abd-49c7-8347-dcb438d90382 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:25.295312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db2b86-6abd-49c7-8347-dcb438d90382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:25.357010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-57db2b86-6abd-49c7-8347-dcb438d90382 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:25.358573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57db2b86-6abd-49c7-8347-dcb438d90382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:25.448407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1359] 10.222.0.55 () {58 vars in 1058 bytes} [Tue May 24 15:30:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:30:25.456121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066ca71-e35a-441e-9402-7c6ce94d0d76 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:25.472314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066ca71-e35a-441e-9402-7c6ce94d0d76 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:25.478786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:25.479962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:25.480413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:25.480850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:25.490517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:25.492529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb75fe-ebb8-4a78-9f94-5ac030414395 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:25.503909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 682/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:27.925709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00640-a662-4f77-b02d-4df39ccdd80c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:27.925709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00640-a662-4f77-b02d-4df39ccdd80c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:27.925709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc00640-a662-4f77-b02d-4df39ccdd80c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:27.930068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1361] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:27.932856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613fdbc-189d-43e8-88b5-c84f9c2e53b6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:27.933324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613fdbc-189d-43e8-88b5-c84f9c2e53b6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:27.933406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7613fdbc-189d-43e8-88b5-c84f9c2e53b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:27.953141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7613fdbc-189d-43e8-88b5-c84f9c2e53b6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:27.954356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7613fdbc-189d-43e8-88b5-c84f9c2e53b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:27.963127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 683/1362] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:28.005851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc00640-a662-4f77-b02d-4df39ccdd80c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:28.022847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc00640-a662-4f77-b02d-4df39ccdd80c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:28.028879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:28.029624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:28.029972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:28.030431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:28.043628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:28.045286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7837cf1-6de1-41e6-a12c-7fc90b1155d1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:28.055393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:30.589370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51f5e-05f2-4824-8ad4-56d9f1fe66a3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:30.589908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51f5e-05f2-4824-8ad4-56d9f1fe66a3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:30.589908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a51f5e-05f2-4824-8ad4-56d9f1fe66a3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:30.591283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 684/1364] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:30.596776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1290b-b9ae-4edf-9fc3-3e05e81da0b3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:30.597562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1290b-b9ae-4edf-9fc3-3e05e81da0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:30.597820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1290b-b9ae-4edf-9fc3-3e05e81da0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:30.620073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e1d1290b-b9ae-4edf-9fc3-3e05e81da0b3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:30.621349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1d1290b-b9ae-4edf-9fc3-3e05e81da0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:30.630136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1365] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:30.679864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a9652-a39a-4a2c-b7da-c9a434edb7a7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:30.681669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a9652-a39a-4a2c-b7da-c9a434edb7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:30.688845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a9652-a39a-4a2c-b7da-c9a434edb7a7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:30.690408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 685/1366] 10.222.0.55 () {54 vars in 961 bytes} [Tue May 24 15:30:30 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:30:30.696867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfda200-0334-40ec-9f97-545ab2a9189e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:30.697726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfda200-0334-40ec-9f97-545ab2a9189e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:30.698182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfda200-0334-40ec-9f97-545ab2a9189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:30.726508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5cfda200-0334-40ec-9f97-545ab2a9189e None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:30.727829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5cfda200-0334-40ec-9f97-545ab2a9189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:30.737282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1367] 10.222.0.55 () {58 vars in 1058 bytes} [Tue May 24 15:30:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:30.745846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a9652-a39a-4a2c-b7da-c9a434edb7a7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:30.760509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a9652-a39a-4a2c-b7da-c9a434edb7a7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:30.765940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:30.766576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:30.766859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:30.767132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:30.775504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:30.777433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052fe293-1fc5-40be-a57c-4dd716eda4a6 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:30.787480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 686/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:30:33.119892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48effe2-1df8-46c8-9097-5f902b217ebf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.121088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48effe2-1df8-46c8-9097-5f902b217ebf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.121596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48effe2-1df8-46c8-9097-5f902b217ebf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.123763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1369] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:33.129091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f84a-6c06-403d-90ab-21c4ff8ab308 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.130020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f84a-6c06-403d-90ab-21c4ff8ab308 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.130459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9f84a-6c06-403d-90ab-21c4ff8ab308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.149304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3ea9f84a-6c06-403d-90ab-21c4ff8ab308 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:33.151089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ea9f84a-6c06-403d-90ab-21c4ff8ab308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:33.166185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 687/1370] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:33.212564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48effe2-1df8-46c8-9097-5f902b217ebf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:33.228471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48effe2-1df8-46c8-9097-5f902b217ebf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:33.235947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:33.237063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.237816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.238311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.248997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:33.250946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6b3f5-c846-4285-a719-2582afae3464 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:33.262509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1371] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:33.525327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2a71-6a9c-459c-897d-b9800ac0913b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.525684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2a71-6a9c-459c-897d-b9800ac0913b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.526138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9e2a71-6a9c-459c-897d-b9800ac0913b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.527710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 688/1372] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:33.533901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49970-775a-4de8-a914-e8f30aaddf6f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.534569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49970-775a-4de8-a914-e8f30aaddf6f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.535058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e49970-775a-4de8-a914-e8f30aaddf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.554833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b8e49970-775a-4de8-a914-e8f30aaddf6f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:33.556634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8e49970-775a-4de8-a914-e8f30aaddf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:33.567453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1373] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:30:33.909584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeefe8-87c0-4e06-b13a-21f439d473e8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.910245 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeefe8-87c0-4e06-b13a-21f439d473e8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.911065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefeefe8-87c0-4e06-b13a-21f439d473e8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.913049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 689/1374] 10.222.0.55 () {54 vars in 963 bytes} [Tue May 24 15:30:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:30:33.921667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6cbf-57bb-4717-a058-9d9ada2ce811 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:33.921991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6cbf-57bb-4717-a058-9d9ada2ce811 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:33.922235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf6cbf-57bb-4717-a058-9d9ada2ce811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:33.931395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:33.987514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fccf6cbf-57bb-4717-a058-9d9ada2ce811 None None] MFA Rules not processed for user `03707ded8dc84180a4ea90986655b84e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:33.989792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fccf6cbf-57bb-4717-a058-9d9ada2ce811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:34.064588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1375] 10.222.0.55 () {58 vars in 1060 bytes} [Tue May 24 15:30:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:34.071908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefeefe8-87c0-4e06-b13a-21f439d473e8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:34.088158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefeefe8-87c0-4e06-b13a-21f439d473e8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:34.094156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:34.094819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:34.095100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:34.095375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:34.107076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:34.108570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b2526-aaba-4bd3-8849-87b84c238758 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:34.121002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 690/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:30:35.549568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c64d9-7ce7-41a9-a2d0-eeddc246f477 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:35.550660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c64d9-7ce7-41a9-a2d0-eeddc246f477 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:35.551080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c64d9-7ce7-41a9-a2d0-eeddc246f477 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:35.552995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1377] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:35.558450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35eab-755b-4b29-a306-c3ed4fd1d259 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:35.558893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35eab-755b-4b29-a306-c3ed4fd1d259 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:35.559011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35eab-755b-4b29-a306-c3ed4fd1d259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:35.580358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2bf35eab-755b-4b29-a306-c3ed4fd1d259 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:35.581801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2bf35eab-755b-4b29-a306-c3ed4fd1d259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:35.595224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 691/1378] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:35.641744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c64d9-7ce7-41a9-a2d0-eeddc246f477 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:35.657301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c64d9-7ce7-41a9-a2d0-eeddc246f477 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:35.665086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:35.665750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:35.666039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:35.667059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:35.676255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:35.678219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c235436-f03d-44b1-a0dc-501445d7704c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:35.691267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:37.985332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac3e2-4944-4667-a9ba-7c1ca4016884 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:37.987914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac3e2-4944-4667-a9ba-7c1ca4016884 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:37.989400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcac3e2-4944-4667-a9ba-7c1ca4016884 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:37.993473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 692/1380] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:38.001198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027052-979c-49a4-a414-20a3d1f8936b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:38.002110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027052-979c-49a4-a414-20a3d1f8936b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:38.002570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46027052-979c-49a4-a414-20a3d1f8936b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:38.039733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-46027052-979c-49a4-a414-20a3d1f8936b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:38.041145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-46027052-979c-49a4-a414-20a3d1f8936b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:38.054579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1381] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:38.111962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcac3e2-4944-4667-a9ba-7c1ca4016884 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:38.140822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcac3e2-4944-4667-a9ba-7c1ca4016884 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:38.152722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:38.153717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:38.154147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:38.154568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:38.164978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:38.166960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b51f63-f8f6-44a0-9f93-6583397c05bf service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:38.178077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 693/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:40.433920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf380-50d3-4631-aa6e-f13ff3bd6dc0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:40.434453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf380-50d3-4631-aa6e-f13ff3bd6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:40.434453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf380-50d3-4631-aa6e-f13ff3bd6dc0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:40.435907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1383] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:40.442031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65786e8-54e4-4269-b1cc-c1778b215dca None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:40.442472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65786e8-54e4-4269-b1cc-c1778b215dca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:40.442581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65786e8-54e4-4269-b1cc-c1778b215dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:40.461032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e65786e8-54e4-4269-b1cc-c1778b215dca None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:40.462219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e65786e8-54e4-4269-b1cc-c1778b215dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:40.470466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 694/1384] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:30:40.512266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cf380-50d3-4631-aa6e-f13ff3bd6dc0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:40.527143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cf380-50d3-4631-aa6e-f13ff3bd6dc0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:40.532774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:40.533621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:40.534081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:40.534532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:40.543721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:40.545585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10fe633-d85d-4519-a415-4590417adf42 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:40.556320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:42.795102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981018f2-e6da-45f3-8a18-4c4965902e9a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:42.795645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981018f2-e6da-45f3-8a18-4c4965902e9a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:42.795645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981018f2-e6da-45f3-8a18-4c4965902e9a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:42.797313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 695/1386] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:42.803723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b5f7-9fb6-42b5-be62-b48576bf8e55 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:42.804200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b5f7-9fb6-42b5-be62-b48576bf8e55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:42.804463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355b5f7-9fb6-42b5-be62-b48576bf8e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:42.825288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1355b5f7-9fb6-42b5-be62-b48576bf8e55 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:42.826651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1355b5f7-9fb6-42b5-be62-b48576bf8e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:42.835497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1387] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:42.884593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981018f2-e6da-45f3-8a18-4c4965902e9a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:42.901836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981018f2-e6da-45f3-8a18-4c4965902e9a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:42.909830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:42.910631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:42.911402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:42.911680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:42.923567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:42.926008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b3e9bb-8cbb-43c7-8cb4-5e881b82d335 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:42.939175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 696/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:45.254910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66222-b074-4169-9205-d65923544cd1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:45.255553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66222-b074-4169-9205-d65923544cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:45.255553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f66222-b074-4169-9205-d65923544cd1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:45.256930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1389] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:45.262333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42df7-f838-4b02-80c7-ec4c311dc55b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:45.262794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42df7-f838-4b02-80c7-ec4c311dc55b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:45.262899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42df7-f838-4b02-80c7-ec4c311dc55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:45.283141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fcc42df7-f838-4b02-80c7-ec4c311dc55b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:45.284795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fcc42df7-f838-4b02-80c7-ec4c311dc55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:45.296806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 697/1390] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:45.343366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f66222-b074-4169-9205-d65923544cd1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:45.358914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f66222-b074-4169-9205-d65923544cd1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:45.365243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:45.365884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:45.366162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:45.366435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:45.375366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:45.377170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaa6cae-acd8-407b-8246-e25daf19c175 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:45.386651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1391] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:47.631765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206dee-4eca-4362-a02b-1a27952e95ea None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:47.632373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206dee-4eca-4362-a02b-1a27952e95ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:47.632495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56206dee-4eca-4362-a02b-1a27952e95ea None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:47.634194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 698/1392] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:47.640315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf240-cf0b-49ac-8112-92b5f75efc82 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:47.640786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf240-cf0b-49ac-8112-92b5f75efc82 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:47.640937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebf240-cf0b-49ac-8112-92b5f75efc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:47.662422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-68ebf240-cf0b-49ac-8112-92b5f75efc82 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:47.664483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-68ebf240-cf0b-49ac-8112-92b5f75efc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:47.674859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1393] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:47.721102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206dee-4eca-4362-a02b-1a27952e95ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:47.741517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56206dee-4eca-4362-a02b-1a27952e95ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:47.749492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:47.750264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:47.750587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:47.750882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:47.761363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:47.763159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349138c-0330-477b-8d21-a82f73db299b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:47.773927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 699/1394] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:50.028221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae21bf6-49c5-4458-b328-efa6008f2f4f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.028861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae21bf6-49c5-4458-b328-efa6008f2f4f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.028861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae21bf6-49c5-4458-b328-efa6008f2f4f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.030129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1395] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:50.035436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113823-4afd-4da2-b4d6-773606649417 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.036251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113823-4afd-4da2-b4d6-773606649417 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.036357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113823-4afd-4da2-b4d6-773606649417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.042816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=52 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:50.058472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-40113823-4afd-4da2-b4d6-773606649417 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:50.059730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-40113823-4afd-4da2-b4d6-773606649417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:50.069112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 700/1396] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:50.116187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae21bf6-49c5-4458-b328-efa6008f2f4f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:50.131216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae21bf6-49c5-4458-b328-efa6008f2f4f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:50.137045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:50.137683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.137960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.138229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.146718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:50.148339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232ae8f-da08-4458-9a9b-812784628043 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:50.158116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1397] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:50.253636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947a9e-cf7f-4f0f-af84-eb19702cca41 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.253983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947a9e-cf7f-4f0f-af84-eb19702cca41 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.254286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98947a9e-cf7f-4f0f-af84-eb19702cca41 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.255778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 701/1398] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:50.262569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92566561-6a48-48a5-be38-aade8a7b0b76 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.262862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92566561-6a48-48a5-be38-aade8a7b0b76 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.263141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92566561-6a48-48a5-be38-aade8a7b0b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.283360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-92566561-6a48-48a5-be38-aade8a7b0b76 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:50.284923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92566561-6a48-48a5-be38-aade8a7b0b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:50.294363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1399] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:50.351901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2928b2-7f1e-4f2a-a5aa-abb20f69c890 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.352195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2928b2-7f1e-4f2a-a5aa-abb20f69c890 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.352511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2928b2-7f1e-4f2a-a5aa-abb20f69c890 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.354028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 702/1400] 10.222.0.55 () {54 vars in 962 bytes} [Tue May 24 15:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:50.359923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f31a6-bf20-4ee3-8658-561e4d6942c4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.360227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f31a6-bf20-4ee3-8658-561e4d6942c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.360504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f31a6-bf20-4ee3-8658-561e4d6942c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.414561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d45f31a6-bf20-4ee3-8658-561e4d6942c4 None None] MFA Rules not processed for user `ae9d48a9a9384ada870d31cdbc9c90e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:50.415765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d45f31a6-bf20-4ee3-8658-561e4d6942c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:50.449110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:50.477326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1401] 10.222.0.55 () {58 vars in 1059 bytes} [Tue May 24 15:30:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:50.485547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2928b2-7f1e-4f2a-a5aa-abb20f69c890 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:50.497832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2928b2-7f1e-4f2a-a5aa-abb20f69c890 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:50.501777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:50.502421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:50.502700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:50.502974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:50.512699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:50.514186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0923a9-5613-455e-9932-16dd6a8b853e service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:50.523970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 703/1402] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:51.063133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842c87-33f9-4872-b811-e5989f0213cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:51.063668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842c87-33f9-4872-b811-e5989f0213cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:51.063780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7842c87-33f9-4872-b811-e5989f0213cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:51.110807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e7842c87-33f9-4872-b811-e5989f0213cf None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:51.111991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7842c87-33f9-4872-b811-e5989f0213cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:51.178372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1403] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:51.219751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177fa97-fdbb-4d30-8466-3787c85fcb24 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:51.220326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177fa97-fdbb-4d30-8466-3787c85fcb24 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:51.220326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177fa97-fdbb-4d30-8466-3787c85fcb24 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:51.221733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 704/1404] 10.222.0.55 () {54 vars in 942 bytes} [Tue May 24 15:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:51.228249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54add3-4a9e-4471-92a3-dd0741074543 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:51.228539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54add3-4a9e-4471-92a3-dd0741074543 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:51.228847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54add3-4a9e-4471-92a3-dd0741074543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:51.278787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bf54add3-4a9e-4471-92a3-dd0741074543 None None] MFA Rules not processed for user `eeb122c4f54840a98cd9fc932fb2856c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:51.280013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bf54add3-4a9e-4471-92a3-dd0741074543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:51.357771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1405] 10.222.0.55 () {58 vars in 1039 bytes} [Tue May 24 15:30:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:51.367190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177fa97-fdbb-4d30-8466-3787c85fcb24 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:51.381301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177fa97-fdbb-4d30-8466-3787c85fcb24 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:51.386859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:51.387624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:51.388011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:51.388396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:51.397734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:51.399490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e4175-4b1f-41d4-bb5d-24e6a04c9b22 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:51.408564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 705/1406] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:30:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:30:52.394593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fa35-adac-404b-8351-eb866e6fadeb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:52.395172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fa35-adac-404b-8351-eb866e6fadeb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:52.395172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0fa35-adac-404b-8351-eb866e6fadeb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:52.396708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1407] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:30:52.401964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d2f7-b4f2-468e-ab1f-e5e069fa1da2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:52.402396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d2f7-b4f2-468e-ab1f-e5e069fa1da2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:52.402508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33d2f7-b4f2-468e-ab1f-e5e069fa1da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:52.422258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-da33d2f7-b4f2-468e-ab1f-e5e069fa1da2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:30:52.423772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-da33d2f7-b4f2-468e-ab1f-e5e069fa1da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:30:52.432939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 706/1408] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:30:52.476669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0fa35-adac-404b-8351-eb866e6fadeb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:52.492138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0fa35-adac-404b-8351-eb866e6fadeb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:52.498785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:52.499538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:52.499893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:52.500279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:52.510205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:52.511636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5470bf8c-ec0c-4ad5-b670-c46fcf8c6ec0 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:52.517076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:52.533857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:02.416962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=17 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:14.360427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91554c07-5510-4478-add3-582b63d17ea3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:14.361173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91554c07-5510-4478-add3-582b63d17ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:14.361173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91554c07-5510-4478-add3-582b63d17ea3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:14.362350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 707/1410] 10.222.0.55 () {54 vars in 918 bytes} [Tue May 24 15:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:14.368600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f4d4-505a-45ef-a879-a841a00c0de2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:14.369108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f4d4-505a-45ef-a879-a841a00c0de2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:14.369108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f4d4-505a-45ef-a879-a841a00c0de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:14.393651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8864f4d4-505a-45ef-a879-a841a00c0de2 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:14.394844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8864f4d4-505a-45ef-a879-a841a00c0de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:14.404046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Tue May 24 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:14.418874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fae44-89a0-4592-9a8b-b2e44fa5d13a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:14.419162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fae44-89a0-4592-9a8b-b2e44fa5d13a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:14.419438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fae44-89a0-4592-9a8b-b2e44fa5d13a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:14.420851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 708/1412] 10.222.0.55 () {54 vars in 918 bytes} [Tue May 24 15:31:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:14.426341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3384-af68-4a35-bdff-78ab707ec051 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:14.426726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3384-af68-4a35-bdff-78ab707ec051 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:14.427068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb3384-af68-4a35-bdff-78ab707ec051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:14.448561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-67bb3384-af68-4a35-bdff-78ab707ec051 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:14.449745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67bb3384-af68-4a35-bdff-78ab707ec051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:14.460524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Tue May 24 15:31:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:17.130778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a28502-d6d8-4925-952f-8c2f093e58d3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:17.131630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a28502-d6d8-4925-952f-8c2f093e58d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:17.131630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a28502-d6d8-4925-952f-8c2f093e58d3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:17.133814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:17.140393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c75c-5ac8-4980-b8cf-edbc948cf149 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:17.141059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c75c-5ac8-4980-b8cf-edbc948cf149 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:17.141216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09c75c-5ac8-4980-b8cf-edbc948cf149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:17.167954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8f09c75c-5ac8-4980-b8cf-edbc948cf149 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:17.169165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f09c75c-5ac8-4980-b8cf-edbc948cf149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:17.177640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:17.220534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a28502-d6d8-4925-952f-8c2f093e58d3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:17.238180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a28502-d6d8-4925-952f-8c2f093e58d3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:17.243945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:17.244704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:17.244986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:17.245280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:17.253247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:17.254827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb46092-8631-4089-b800-e20be4ca00a0 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:17.264818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 710/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:18.242057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c23fb2-1b73-4144-a816-6b6da5d0b042 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:18.242923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c23fb2-1b73-4144-a816-6b6da5d0b042 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:18.242923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c23fb2-1b73-4144-a816-6b6da5d0b042 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:18.244953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:18.250914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6c4-92c3-4bdd-892a-64554afd35f4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:18.251354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6c4-92c3-4bdd-892a-64554afd35f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:18.251471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c6c4-92c3-4bdd-892a-64554afd35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:18.277326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b738c6c4-92c3-4bdd-892a-64554afd35f4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:18.278487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b738c6c4-92c3-4bdd-892a-64554afd35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:18.287880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:18.447756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c23fb2-1b73-4144-a816-6b6da5d0b042 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:18.469218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c23fb2-1b73-4144-a816-6b6da5d0b042 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:18.475358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:18.475991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:18.476307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:18.476579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:18.484986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:18.486589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b7b22-9a4e-4cd7-9258-2363614aca78 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:18.496213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:19.425044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ead38c-183b-40ab-9e1c-f5be2800ce99 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:19.425600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ead38c-183b-40ab-9e1c-f5be2800ce99 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:19.425600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ead38c-183b-40ab-9e1c-f5be2800ce99 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:19.427679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:19.434624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0f876-8ec1-4d60-a112-195d02046b7b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:19.435146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0f876-8ec1-4d60-a112-195d02046b7b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:19.435264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0f876-8ec1-4d60-a112-195d02046b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:19.456569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90a0f876-8ec1-4d60-a112-195d02046b7b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:19.457771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90a0f876-8ec1-4d60-a112-195d02046b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:19.468471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:19.638335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ead38c-183b-40ab-9e1c-f5be2800ce99 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:19.663828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ead38c-183b-40ab-9e1c-f5be2800ce99 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:19.671270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:19.671937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:19.672241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:19.672511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:19.681784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:19.683408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f0c9e-81ae-4781-975a-38ce33138a2a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:19.697010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 713/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:20.625162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeefa2d-2785-4368-ae9f-986ec117a79c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:20.625682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeefa2d-2785-4368-ae9f-986ec117a79c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:20.625682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeefa2d-2785-4368-ae9f-986ec117a79c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:20.627696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:20.635800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946350-68d7-44f7-b6a6-6cb3a16ac816 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:20.636311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946350-68d7-44f7-b6a6-6cb3a16ac816 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:20.636404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91946350-68d7-44f7-b6a6-6cb3a16ac816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:20.659276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-91946350-68d7-44f7-b6a6-6cb3a16ac816 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:20.660527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-91946350-68d7-44f7-b6a6-6cb3a16ac816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:20.670307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:31:20.822825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeefa2d-2785-4368-ae9f-986ec117a79c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:20.839014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeefa2d-2785-4368-ae9f-986ec117a79c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:20.845011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:20.845645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:20.845922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:20.846195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:20.855332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:20.856970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c9ad36-ab1c-4b76-8a6a-a9901826c086 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:20.866916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:21.788805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555fe2-17eb-4764-8bf3-cdfd75c33537 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:21.789390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555fe2-17eb-4764-8bf3-cdfd75c33537 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:21.789390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e555fe2-17eb-4764-8bf3-cdfd75c33537 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:21.790730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:21.797230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a844-1638-44dd-ba13-189774b70b05 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:21.797908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a844-1638-44dd-ba13-189774b70b05 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:21.798055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7a844-1638-44dd-ba13-189774b70b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:21.823164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e7b7a844-1638-44dd-ba13-189774b70b05 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:21.824376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7b7a844-1638-44dd-ba13-189774b70b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:21.832734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:21.981180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555fe2-17eb-4764-8bf3-cdfd75c33537 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:21.997643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e555fe2-17eb-4764-8bf3-cdfd75c33537 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:22.007671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:22.008651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:22.008993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:22.009284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:22.023622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:22.026063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec1dec3-07ba-4e21-bb45-e6c05a4a28cf service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:22.035955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 716/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:22.879459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24fb4-0606-4867-a92c-529280bd15ce None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:22.880022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24fb4-0606-4867-a92c-529280bd15ce None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:22.880022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a24fb4-0606-4867-a92c-529280bd15ce None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:22.881423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:22.887540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8891-1cd6-408a-b6c9-b83ed78d5f73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:22.888227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8891-1cd6-408a-b6c9-b83ed78d5f73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:22.889674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe8891-1cd6-408a-b6c9-b83ed78d5f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:22.913649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c2fe8891-1cd6-408a-b6c9-b83ed78d5f73 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:22.914861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2fe8891-1cd6-408a-b6c9-b83ed78d5f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:22.923978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:23.075231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24fb4-0606-4867-a92c-529280bd15ce None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:23.100551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a24fb4-0606-4867-a92c-529280bd15ce None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:23.108141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:23.108780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:23.109059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:23.109334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:23.117650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:23.119252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980c4bf-01d8-41a8-a581-f61fc1171549 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:23.128695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:23.970828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504f6c-4391-4761-bc40-fcde72e581c0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:23.971401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504f6c-4391-4761-bc40-fcde72e581c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:23.971401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61504f6c-4391-4761-bc40-fcde72e581c0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:23.973512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:23.981198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3601b7-4e34-416f-80a7-1a2e8d2634e4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:23.981586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3601b7-4e34-416f-80a7-1a2e8d2634e4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:23.981740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3601b7-4e34-416f-80a7-1a2e8d2634e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:24.001793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9e3601b7-4e34-416f-80a7-1a2e8d2634e4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:24.002979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9e3601b7-4e34-416f-80a7-1a2e8d2634e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:24.012017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:24.163494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61504f6c-4391-4761-bc40-fcde72e581c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:24.180664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61504f6c-4391-4761-bc40-fcde72e581c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:24.186657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:24.187293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:24.187568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:24.187843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:24.196168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:24.197897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee593fe1-eb0f-4c58-a053-f557def38b9d service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:24.207460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 719/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:25.058264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2cf75-78a8-4318-99bd-adcace66eec6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:25.059169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2cf75-78a8-4318-99bd-adcace66eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:25.059169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2cf75-78a8-4318-99bd-adcace66eec6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:25.061338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:25.067441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85cb3-438c-4c4a-98ef-2538d5f0df10 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:25.068167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85cb3-438c-4c4a-98ef-2538d5f0df10 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:25.068317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e85cb3-438c-4c4a-98ef-2538d5f0df10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:25.094969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c6e85cb3-438c-4c4a-98ef-2538d5f0df10 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:25.096224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6e85cb3-438c-4c4a-98ef-2538d5f0df10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:25.105281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:25.264220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2cf75-78a8-4318-99bd-adcace66eec6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:25.285435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2cf75-78a8-4318-99bd-adcace66eec6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:25.292905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:25.293832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:25.294201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:25.294627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:25.304787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:25.306395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa692c9-3725-4bf7-8c44-80ae54b64413 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:25.318157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:26.165329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f1d8-f39c-4b01-9eba-0711458fa815 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:26.166344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f1d8-f39c-4b01-9eba-0711458fa815 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:26.166344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f1d8-f39c-4b01-9eba-0711458fa815 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:26.167668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:26.172949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedc17-84f1-4714-8a04-87a733963f82 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:26.173366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedc17-84f1-4714-8a04-87a733963f82 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:26.173475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdedc17-84f1-4714-8a04-87a733963f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:26.193327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1bdedc17-84f1-4714-8a04-87a733963f82 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:26.194513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1bdedc17-84f1-4714-8a04-87a733963f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:26.206743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:26.357057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f1d8-f39c-4b01-9eba-0711458fa815 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:26.382786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f1d8-f39c-4b01-9eba-0711458fa815 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:26.390247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:26.390972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:26.391288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:26.391532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:26.399758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:26.401376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fb3e1-9d97-4a04-b19d-c5f9519c7ebd service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:26.411236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 722/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:28.465211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:28.466067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:28.466067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:28.468392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:28.473803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53a427-406a-4628-af40-b9cf819d84a1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:28.474248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53a427-406a-4628-af40-b9cf819d84a1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:28.474352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53a427-406a-4628-af40-b9cf819d84a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:28.492769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cd53a427-406a-4628-af40-b9cf819d84a1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:28.493945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd53a427-406a-4628-af40-b9cf819d84a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:28.503410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:28.542554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:28.544454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:28.558684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9ed1-b647-4790-9231-38d958ecbcb2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:28.564918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:28.565840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:28.566291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:28.566714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:28.578964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:28.580452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1728a5-a21d-4e32-846d-a75e4826a4ef service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:28.590676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:30.902597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857cde-0fe3-4a49-88b2-f93df55ba025 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:30.903304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857cde-0fe3-4a49-88b2-f93df55ba025 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:30.903304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60857cde-0fe3-4a49-88b2-f93df55ba025 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:30.904542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:30.911105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002dc5-2bbe-43b9-8461-b8f4705098e9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:30.911536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002dc5-2bbe-43b9-8461-b8f4705098e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:30.911648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4002dc5-2bbe-43b9-8461-b8f4705098e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:30.933727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4002dc5-2bbe-43b9-8461-b8f4705098e9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:30.934919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4002dc5-2bbe-43b9-8461-b8f4705098e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:30.943496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:30.980882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60857cde-0fe3-4a49-88b2-f93df55ba025 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:30.994382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60857cde-0fe3-4a49-88b2-f93df55ba025 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:30.998692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:30.999341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:30.999604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:30.999876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:31.009220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:31.010827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ec429-8ac8-40d7-87db-520d3e898e04 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:31.020372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 725/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:33.605187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee7110-2165-45c3-b13b-cd4308f5f254 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:33.605780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee7110-2165-45c3-b13b-cd4308f5f254 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:33.605780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee7110-2165-45c3-b13b-cd4308f5f254 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:33.607390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:33.613863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93dd-77ae-4758-be0a-353ba427f909 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:33.614617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93dd-77ae-4758-be0a-353ba427f909 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:33.614799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a93dd-77ae-4758-be0a-353ba427f909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:33.641689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-811a93dd-77ae-4758-be0a-353ba427f909 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:33.642882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-811a93dd-77ae-4758-be0a-353ba427f909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:33.651837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:33.695162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee7110-2165-45c3-b13b-cd4308f5f254 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:33.707695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee7110-2165-45c3-b13b-cd4308f5f254 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:33.711961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:33.712631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:33.712880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:33.713148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:33.722987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:33.724484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b0933-4a06-4af0-87ad-3b3abc8f4cd7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:33.734130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:36.107543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f9677-c583-4603-8dda-6dc62357cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:36.108222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f9677-c583-4603-8dda-6dc62357cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:36.108222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f9677-c583-4603-8dda-6dc62357cdf8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:36.109495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:36.115979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28912f-798f-493c-808c-97c96ae3ee12 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:36.116837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28912f-798f-493c-808c-97c96ae3ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:36.117033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28912f-798f-493c-808c-97c96ae3ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:36.143654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-da28912f-798f-493c-808c-97c96ae3ee12 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:36.144871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-da28912f-798f-493c-808c-97c96ae3ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:36.153213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:36.195113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f9677-c583-4603-8dda-6dc62357cdf8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:36.214516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f9677-c583-4603-8dda-6dc62357cdf8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:36.219012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:36.219648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:36.219925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:36.220200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:36.230072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:36.231519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bd3bb-2323-47ed-ac9f-33a3c5241331 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:36.240938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 728/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:38.467220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856dbd-bf63-49ac-a1d0-98e805ca8dfb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:38.467811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856dbd-bf63-49ac-a1d0-98e805ca8dfb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:38.467811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34856dbd-bf63-49ac-a1d0-98e805ca8dfb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:38.469185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:31:38.475678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6404-c913-40f2-9b62-37bf9ddb77ae None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:38.476459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6404-c913-40f2-9b62-37bf9ddb77ae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:38.476621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac6404-c913-40f2-9b62-37bf9ddb77ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:38.501929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-81ac6404-c913-40f2-9b62-37bf9ddb77ae None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:38.503167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-81ac6404-c913-40f2-9b62-37bf9ddb77ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:38.512644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:38.553456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856dbd-bf63-49ac-a1d0-98e805ca8dfb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:38.567074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34856dbd-bf63-49ac-a1d0-98e805ca8dfb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:38.571495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:38.572163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:38.572587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:38.572992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:38.582215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:38.583692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e00320-0fe9-4e47-8823-049d8e9befe3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:38.593480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:40.892717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232b91-8864-4b2b-8e94-71c7ec14868f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:40.893309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232b91-8864-4b2b-8e94-71c7ec14868f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:40.893309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232b91-8864-4b2b-8e94-71c7ec14868f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:40.894622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:40.901424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53d90-c3af-416c-bd3c-d907e2827285 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:40.902062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53d90-c3af-416c-bd3c-d907e2827285 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:40.902172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a53d90-c3af-416c-bd3c-d907e2827285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:41.050318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b9a53d90-c3af-416c-bd3c-d907e2827285 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:41.051529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9a53d90-c3af-416c-bd3c-d907e2827285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:41.060752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:41.097734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac232b91-8864-4b2b-8e94-71c7ec14868f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:41.110775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac232b91-8864-4b2b-8e94-71c7ec14868f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:41.114712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:41.115374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:41.115649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:41.115916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:41.125990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:41.127432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f273fe-e3bf-4b1f-a1e6-f92e451e1707 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:41.136907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 731/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:43.362688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c63-df63-45cf-8e1c-b9f2711b7f6e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:43.363303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c63-df63-45cf-8e1c-b9f2711b7f6e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:43.363303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c1c63-df63-45cf-8e1c-b9f2711b7f6e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:43.364618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:43.371232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af1e6-a8cc-40c0-a586-c6427a874d26 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:43.371950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af1e6-a8cc-40c0-a586-c6427a874d26 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:43.372176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230af1e6-a8cc-40c0-a586-c6427a874d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:43.382587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=70 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:43.396579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-230af1e6-a8cc-40c0-a586-c6427a874d26 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:43.397791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-230af1e6-a8cc-40c0-a586-c6427a874d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:43.407244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:43.447830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1c63-df63-45cf-8e1c-b9f2711b7f6e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:43.455698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=73 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:43.462634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c1c63-df63-45cf-8e1c-b9f2711b7f6e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:43.466450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:43.467073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:43.467345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:43.467614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:43.477458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:43.479195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcf946-9646-46d3-ab8b-44ef8456dff7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:43.488540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:45.709025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf01c1-31c2-482b-93a6-db5b78e771b2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:45.709527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf01c1-31c2-482b-93a6-db5b78e771b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:45.709527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf01c1-31c2-482b-93a6-db5b78e771b2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:45.711494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:45.717013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8838-1545-4a27-9c28-a19a30fb50b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:45.717449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8838-1545-4a27-9c28-a19a30fb50b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:45.717559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e8838-1545-4a27-9c28-a19a30fb50b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:45.736078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-525e8838-1545-4a27-9c28-a19a30fb50b4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:45.737253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-525e8838-1545-4a27-9c28-a19a30fb50b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:45.745749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:45.782164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf01c1-31c2-482b-93a6-db5b78e771b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:45.795517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf01c1-31c2-482b-93a6-db5b78e771b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:45.799873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:45.800558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:45.800838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:45.801109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:45.810647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:45.812114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5146a-0973-4d63-a3be-6488f67392be service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:45.821751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 734/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:48.034480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d267c-a826-4bca-a6ee-936fe243bda4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:48.035068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d267c-a826-4bca-a6ee-936fe243bda4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:48.035068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d267c-a826-4bca-a6ee-936fe243bda4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:48.037623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:48.044773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2987-fc59-41ea-8262-9a7b141d20fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:48.045242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2987-fc59-41ea-8262-9a7b141d20fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:48.045337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d2987-fc59-41ea-8262-9a7b141d20fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:48.064264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a76d2987-fc59-41ea-8262-9a7b141d20fd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:48.065466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a76d2987-fc59-41ea-8262-9a7b141d20fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:48.076383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:31:48.115722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d267c-a826-4bca-a6ee-936fe243bda4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:48.135027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d267c-a826-4bca-a6ee-936fe243bda4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:48.139471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:48.140143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:48.140418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:48.140698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:48.150155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:48.151600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec9b47-65cf-41da-82b1-e43c0003bf90 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:48.161723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:50.378431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab2996-8c39-4098-987b-3507ec5bdfba None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:50.378996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab2996-8c39-4098-987b-3507ec5bdfba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:50.378996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab2996-8c39-4098-987b-3507ec5bdfba None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:50.381209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:31:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:31:50.389269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000422a6-2b92-4da7-bd22-369c184f9cc9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:50.389676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000422a6-2b92-4da7-bd22-369c184f9cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:50.389863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000422a6-2b92-4da7-bd22-369c184f9cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:50.417756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-000422a6-2b92-4da7-bd22-369c184f9cc9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:31:50.418948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-000422a6-2b92-4da7-bd22-369c184f9cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:50.427871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:31:50.464531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab2996-8c39-4098-987b-3507ec5bdfba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:50.483397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab2996-8c39-4098-987b-3507ec5bdfba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:50.488159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:50.488802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:50.489072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:50.489348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:50.499556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:50.501054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be1c39-504d-428d-bd87-68500f16a2c6 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:50.510852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 737/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:00.410111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=17 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:32:00.506491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=17 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:32:23.020904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0275f82-0925-4f38-a66c-445826cdf229 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:23.021533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0275f82-0925-4f38-a66c-445826cdf229 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:23.021533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0275f82-0925-4f38-a66c-445826cdf229 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:23.023502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1471] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:23.029713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38034-9149-4574-bf0d-05e423ff0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:23.030212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38034-9149-4574-bf0d-05e423ff0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:23.030212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38034-9149-4574-bf0d-05e423ff0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:23.048739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5d38034-9149-4574-bf0d-05e423ff0ff7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:23.049915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5d38034-9149-4574-bf0d-05e423ff0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:23.058142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 738/1472] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:32:23.110587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71382-dc6f-41cd-8e78-8ab1d672f562 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:23.110864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71382-dc6f-41cd-8e78-8ab1d672f562 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:23.111129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71382-dc6f-41cd-8e78-8ab1d672f562 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:23.112497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1473] 10.222.0.55 () {54 vars in 962 bytes} [Tue May 24 15:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:23.117422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711969-2a96-4b73-82d6-72c2f64ec92d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:23.117708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711969-2a96-4b73-82d6-72c2f64ec92d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:23.117955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711969-2a96-4b73-82d6-72c2f64ec92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:23.138038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d3711969-2a96-4b73-82d6-72c2f64ec92d None None] MFA Rules not processed for user `ae9d48a9a9384ada870d31cdbc9c90e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:23.139238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d3711969-2a96-4b73-82d6-72c2f64ec92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:23.146106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 739/1474] 10.222.0.55 () {58 vars in 1059 bytes} [Tue May 24 15:32:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:23.153523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71382-dc6f-41cd-8e78-8ab1d672f562 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:23.165313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71382-dc6f-41cd-8e78-8ab1d672f562 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:23.169759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:23.170501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:23.170773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:23.171041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:23.180339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:23.181774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5d4a4-c713-42eb-a436-bdea71705bdc service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:23.190973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:25.441148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd40d-5dfc-4012-a654-de8d5156c4c9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:25.441788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd40d-5dfc-4012-a654-de8d5156c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:25.441788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3fd40d-5dfc-4012-a654-de8d5156c4c9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:25.443163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 740/1476] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:25.449235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cb6d9-5fc3-43fa-b790-3b6c1443808f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:25.449702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cb6d9-5fc3-43fa-b790-3b6c1443808f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:25.449897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cb6d9-5fc3-43fa-b790-3b6c1443808f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:25.477029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-607cb6d9-5fc3-43fa-b790-3b6c1443808f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:25.478658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-607cb6d9-5fc3-43fa-b790-3b6c1443808f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:25.487119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1477] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:25.511946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fd40d-5dfc-4012-a654-de8d5156c4c9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:25.534333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3fd40d-5dfc-4012-a654-de8d5156c4c9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:25.542153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:25.542842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:25.543122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:25.543393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:25.544177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:25.545963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:32:25.545963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:32:25.545963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:32:25.547117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca5777-1492-4c71-aecd-2aaa05bac82a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:25.567138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 741/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Tue May 24 15:32:25 2022] GET /identity/v3/projects => generated 2553 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:32:27.757165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391423-4075-40a1-8283-677e1d27ee81 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:27.757718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391423-4075-40a1-8283-677e1d27ee81 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:27.757718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23391423-4075-40a1-8283-677e1d27ee81 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:27.759095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1479] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:27.764491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a417-2c7c-4063-8ac8-6de100b4e713 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:27.764941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a417-2c7c-4063-8ac8-6de100b4e713 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:27.765046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73a417-2c7c-4063-8ac8-6de100b4e713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:27.783797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f73a417-2c7c-4063-8ac8-6de100b4e713 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:27.785010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f73a417-2c7c-4063-8ac8-6de100b4e713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:27.793815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 742/1480] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:27.836192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23391423-4075-40a1-8283-677e1d27ee81 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:27.856981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23391423-4075-40a1-8283-677e1d27ee81 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:27.862608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:27.863251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:27.863528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:27.863797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:27.872921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:27.874564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef4fb0-ad5f-4e34-aac7-973638182ee8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:27.884064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:30.282679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550abb3-ae17-456d-9d6f-dd90113c9043 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:30.283199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550abb3-ae17-456d-9d6f-dd90113c9043 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:30.283199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550abb3-ae17-456d-9d6f-dd90113c9043 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:30.284727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 743/1482] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:30.290516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521954d-62d9-47d9-b915-b72a9b217210 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:30.290998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521954d-62d9-47d9-b915-b72a9b217210 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:30.291152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521954d-62d9-47d9-b915-b72a9b217210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:30.309897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7521954d-62d9-47d9-b915-b72a9b217210 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:30.311074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7521954d-62d9-47d9-b915-b72a9b217210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:30.320392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1483] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:30.357484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550abb3-ae17-456d-9d6f-dd90113c9043 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:30.369207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550abb3-ae17-456d-9d6f-dd90113c9043 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:30.373786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:30.374415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:30.374691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:30.374971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:30.388405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:30.390696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee7eda-f91b-42ad-ab4b-6c9b2ab3e1f8 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:30.403706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 744/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:32.648330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e4825-cd3f-426a-b3a0-256ec6f8d2a6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:32.649175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e4825-cd3f-426a-b3a0-256ec6f8d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:32.649175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e4825-cd3f-426a-b3a0-256ec6f8d2a6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:32.650916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1485] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:32.657395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda6d1-63a8-4ef3-a2b1-222ee49bd26a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:32.657846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda6d1-63a8-4ef3-a2b1-222ee49bd26a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:32.657961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deda6d1-63a8-4ef3-a2b1-222ee49bd26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:32.679380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4deda6d1-63a8-4ef3-a2b1-222ee49bd26a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:32.680601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4deda6d1-63a8-4ef3-a2b1-222ee49bd26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:32.689723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 745/1486] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:32.727126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e4825-cd3f-426a-b3a0-256ec6f8d2a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:32.738122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e4825-cd3f-426a-b3a0-256ec6f8d2a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:32.742171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:32.742818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:32.743093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:32.743359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:32.753119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:32.754610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80760a-1f75-4541-b87c-10ed5c56b232 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:32.764352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:32:35.104776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be090fa-65ec-4bc5-94fd-a7afdbf761bb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:35.105641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be090fa-65ec-4bc5-94fd-a7afdbf761bb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:35.105641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be090fa-65ec-4bc5-94fd-a7afdbf761bb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:35.107789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 746/1488] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:35.116884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1e270-6aad-4e6c-8179-44965c24b352 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:35.117311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1e270-6aad-4e6c-8179-44965c24b352 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:35.117450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1e270-6aad-4e6c-8179-44965c24b352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:35.142720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5cb1e270-6aad-4e6c-8179-44965c24b352 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:35.143909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5cb1e270-6aad-4e6c-8179-44965c24b352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:35.155108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1489] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:35.199114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be090fa-65ec-4bc5-94fd-a7afdbf761bb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:35.211322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be090fa-65ec-4bc5-94fd-a7afdbf761bb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:35.215852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:35.216452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:35.216759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:35.217028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:35.226638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:35.228139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd33b9-c088-4e85-8d5b-750586a9c7da service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:35.237496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 747/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:37.633681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b6c-e0fc-40c3-9666-bb0ee5c01a35 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:37.634580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b6c-e0fc-40c3-9666-bb0ee5c01a35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:37.634580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b6c-e0fc-40c3-9666-bb0ee5c01a35 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:37.635876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1491] 10.222.0.55 () {54 vars in 926 bytes} [Tue May 24 15:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:32:37.641369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af8ab-cb65-4d32-88d4-fc4ae5a72c9c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:37.641844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af8ab-cb65-4d32-88d4-fc4ae5a72c9c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:37.641959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52af8ab-cb65-4d32-88d4-fc4ae5a72c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:37.661974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e52af8ab-cb65-4d32-88d4-fc4ae5a72c9c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:32:37.663170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e52af8ab-cb65-4d32-88d4-fc4ae5a72c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:32:37.672600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 748/1492] 10.222.0.55 () {58 vars in 1023 bytes} [Tue May 24 15:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:32:37.711539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594b6c-e0fc-40c3-9666-bb0ee5c01a35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:32:37.724365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594b6c-e0fc-40c3-9666-bb0ee5c01a35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:32:37.728346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:32:37.728970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:32:37.729255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:32:37.729521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:32:37.738625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:32:37.740056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab0214-ee10-47fe-b337-46123010bb0e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:32:37.749634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:32:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:32:47.655801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:32:47.745418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:34:19.037469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.038080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.038080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.058710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:34:19.059933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:19.070274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 749/1494] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:34:19.079885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.080462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.080462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.099449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:34:19.100654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:19.109725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1495] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:34:19.114295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.123913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdf2d1-45ef-4666-a372-33f6dfe2f54f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.128812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.129457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.129731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.129991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.130706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:34:19.131006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.132051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.132051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:34:19.132051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:34:19.132959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.144489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 750/1496] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:34:19.153119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.167622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baac87a-05e6-453b-abe7-a04041e46bef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.176572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.177481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.177841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.178170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.179053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.180951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.180951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:34:19.180951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:34:19.182125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.221436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1497] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:34:19.225810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.235781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36b476-ca0e-4b06-8fb0-6e8d3249fe7b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.293485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.294082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.294360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.294624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.295335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.296397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.296397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:34:19.296397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:34:19.297603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.321568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 751/1498] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:34:19.325809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.336225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5887408-b1fa-4c92-8906-45036201bba9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.341148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.341842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.342115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.342389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.343129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.344202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.344202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:34:19.344202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:34:19.345076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.358463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1499] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:34:19.362715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.373645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c9cf-c475-4fe0-9dae-8b4bda4bbc67 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.379322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.380002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.380305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.380572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.381285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.382290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.382290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:34:19.382290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:34:19.383209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd67bf4d-0b86-4fef-949c-025b9c65e510 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.397029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 752/1500] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:34:19.401638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.412153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d6b79-7fc4-4e50-a180-f523bd0e6eb3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.417601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.418493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.418769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.419039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] PATH_INFO: `/v3/projects/4c58dbf9af274425abf29e3ba836de88/users/7ada829a46004216b0c1bd15d1c3024c/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.433588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c58dbf9af274425abf29e3ba836de88, user_id=7ada829a46004216b0c1bd15d1c3024c, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.434638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.434638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:34:19.434638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:34:19.436627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d56d7-9edb-4e5b-8ab0-528ca12b7673 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.451536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1501] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:34:19 2022] PUT /identity/v3/projects/4c58dbf9af274425abf29e3ba836de88/users/7ada829a46004216b0c1bd15d1c3024c/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:34:19.466873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e8aa-2743-474c-ba81-c9f1c95d69da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.467159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e8aa-2743-474c-ba81-c9f1c95d69da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.467434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506e8aa-2743-474c-ba81-c9f1c95d69da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.494200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2506e8aa-2743-474c-ba81-c9f1c95d69da None None] MFA Rules not processed for user `7ada829a46004216b0c1bd15d1c3024c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:34:19.495469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2506e8aa-2743-474c-ba81-c9f1c95d69da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:19.560247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 753/1502] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:34:19.574585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be92fa-1bbb-4e10-aaec-79d359734d8a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.574871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be92fa-1bbb-4e10-aaec-79d359734d8a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.575170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4be92fa-1bbb-4e10-aaec-79d359734d8a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.577294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1503] 10.222.0.55 () {54 vars in 963 bytes} [Tue May 24 15:34:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:34:19.583717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c7f12-215f-41b5-a1b0-31f340773afe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.583999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c7f12-215f-41b5-a1b0-31f340773afe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.584297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c7f12-215f-41b5-a1b0-31f340773afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.602852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-242c7f12-215f-41b5-a1b0-31f340773afe None None] MFA Rules not processed for user `03707ded8dc84180a4ea90986655b84e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:34:19.604066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-242c7f12-215f-41b5-a1b0-31f340773afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:19.647903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:34:19.659866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 754/1504] 10.222.0.55 () {58 vars in 1060 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:34:19.668987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be92fa-1bbb-4e10-aaec-79d359734d8a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.680333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4be92fa-1bbb-4e10-aaec-79d359734d8a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.683979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.684643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.684919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.685187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.694305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.695802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.704829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:34:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:34:19.747254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd1b56-bd0b-4c92-ae38-ed3a3e0c40fc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.747536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd1b56-bd0b-4c92-ae38-ed3a3e0c40fc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.747820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd1b56-bd0b-4c92-ae38-ed3a3e0c40fc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.749132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 755/1506] 10.222.0.55 () {52 vars in 829 bytes} [Tue May 24 15:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:34:19.752860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:19.820129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7588e-8dda-42b9-b452-b5ed6c148840 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:19.825143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:19.826016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.826287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.826557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] PATH_INFO: `/v3/users/7ada829a46004216b0c1bd15d1c3024c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.827799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ada829a46004216b0c1bd15d1c3024c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:19.828863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:19.828863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:34:19.828863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:34:19.830050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:19.898804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ada829a46004216b0c1bd15d1c3024c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:19.909508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ada829a46004216b0c1bd15d1c3024c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:19.910125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ada829a46004216b0c1bd15d1c3024c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:19.910866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ada829a46004216b0c1bd15d1c3024c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:19.930244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ada829a46004216b0c1bd15d1c3024c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:19.938802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:34:19.952551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1507] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:34:19 2022] DELETE /identity/v3/users/7ada829a46004216b0c1bd15d1c3024c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:34:19.957087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f166-485d-4b16-a037-1b503182165a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:19.957377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f166-485d-4b16-a037-1b503182165a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:19.957656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172f166-485d-4b16-a037-1b503182165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:19.977466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6172f166-485d-4b16-a037-1b503182165a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:34:19.978705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6172f166-485d-4b16-a037-1b503182165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:20.039457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 756/1508] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:34:20.049075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:20.050545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:20.111735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d0e95-6471-4eb5-a799-dfd22bf8361b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:20.116499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:20.117251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:20.117634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:20.118060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:20.133237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:20.134710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d60f34-6def-4f46-9596-264a28e78bfb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:20.144824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:34:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:34:20.497196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172f166-485d-4b16-a037-1b503182165a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:34:20.498738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6172f166-485d-4b16-a037-1b503182165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:34:20.510766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172f166-485d-4b16-a037-1b503182165a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:34:20.516241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:34:20.517524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:34:20.517789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:34:20.518221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] PATH_INFO: `/v3/projects/4c58dbf9af274425abf29e3ba836de88` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:34:20.519284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c58dbf9af274425abf29e3ba836de88)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:34:20.520813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:34:20.520813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:34:20.520813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:34:20.522391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:34:20.589057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c58dbf9af274425abf29e3ba836de88', 'request_id': 'req-402200e4-466e-450d-9448-6c115f2e5a7d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:20.595549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c58dbf9af274425abf29e3ba836de88', 'request_id': 'req-402200e4-466e-450d-9448-6c115f2e5a7d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:20.606462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c58dbf9af274425abf29e3ba836de88', 'request_id': 'req-402200e4-466e-450d-9448-6c115f2e5a7d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:20.625829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] The token cache is being invalidate because project 4c58dbf9af274425abf29e3ba836de88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:34:20.626009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:20.626847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-402200e4-466e-450d-9448-6c115f2e5a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:34:20.628540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 757/1510] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:34:20 2022] DELETE /identity/v3/projects/4c58dbf9af274425abf29e3ba836de88 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:34:30.140214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=29 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:34:30.622461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:01.528169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:01.528838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:01.528838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:01.530100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:36:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:01.536399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9d7e-50fb-44ca-b259-560652c84996 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:01.537150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9d7e-50fb-44ca-b259-560652c84996 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:01.537334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d9d7e-50fb-44ca-b259-560652c84996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:01.563150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f35d9d7e-50fb-44ca-b259-560652c84996 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:01.564415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f35d9d7e-50fb-44ca-b259-560652c84996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:01.628336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:01.814228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:01.817493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:01.896878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef07349-06bd-49bd-aa28-2168c4ccf7ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:01.903386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:01.904073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:01.904372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:01.904643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:01.915156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:01.916783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4101a-b7f1-4f17-aaec-d8f9b1d70129 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:01.926656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:02.400569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47983-e61e-432d-8d2c-9b4315020ed6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:02.400841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47983-e61e-432d-8d2c-9b4315020ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:02.401139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47983-e61e-432d-8d2c-9b4315020ed6 None None] PATH_INFO: `/v3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:02.402466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 759/1514] 10.222.0.59 () {54 vars in 925 bytes} [Tue May 24 15:36:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:36:02.410163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49268-3509-4800-9865-257fd4da14e6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:02.410417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49268-3509-4800-9865-257fd4da14e6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:02.410720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd49268-3509-4800-9865-257fd4da14e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:02.431266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7fd49268-3509-4800-9865-257fd4da14e6 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:02.432582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7fd49268-3509-4800-9865-257fd4da14e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:02.501251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1515] 10.222.0.59 () {58 vars in 1015 bytes} [Tue May 24 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:02.799127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3772c-10a9-4e10-b172-829f73ef20ed None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:02.799725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3772c-10a9-4e10-b172-829f73ef20ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:02.799725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3772c-10a9-4e10-b172-829f73ef20ed None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:02.802326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:36:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:02.808926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82f896-2f1d-4665-bcb9-21ca04e20e43 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:02.809379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82f896-2f1d-4665-bcb9-21ca04e20e43 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:02.809492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82f896-2f1d-4665-bcb9-21ca04e20e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:02.828906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5a82f896-2f1d-4665-bcb9-21ca04e20e43 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:02.830078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a82f896-2f1d-4665-bcb9-21ca04e20e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:02.838272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:02.991318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3772c-10a9-4e10-b172-829f73ef20ed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:03.008600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3772c-10a9-4e10-b172-829f73ef20ed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:03.015260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:03.015943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:03.016251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:03.016522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:03.025013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:03.026673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3ec29-6d4a-48b9-8629-bf2d82716659 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:03.037293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 761/1518] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:11.265432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c372c5-265a-47a6-b7ac-5a8a6cfc689f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:11.266242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c372c5-265a-47a6-b7ac-5a8a6cfc689f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:11.266600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c372c5-265a-47a6-b7ac-5a8a6cfc689f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:11.268898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1519] 10.222.0.55 () {54 vars in 960 bytes} [Tue May 24 15:36:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:11.274033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b940ab-ec59-4255-ab7e-1cd58fa286a8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:11.274501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b940ab-ec59-4255-ab7e-1cd58fa286a8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:11.274596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b940ab-ec59-4255-ab7e-1cd58fa286a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:11.294335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-94b940ab-ec59-4255-ab7e-1cd58fa286a8 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:11.295542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94b940ab-ec59-4255-ab7e-1cd58fa286a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:11.304206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 762/1520] 10.222.0.55 () {58 vars in 1057 bytes} [Tue May 24 15:36:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:11.311652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c372c5-265a-47a6-b7ac-5a8a6cfc689f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:11.325130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c372c5-265a-47a6-b7ac-5a8a6cfc689f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:11.329903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:11.330533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:11.330809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:11.331084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:11.339353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:11.340984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7784e-74d3-4988-b3c7-8499dc369ce9 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:11.350731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:14.312776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd9212-a6e3-4f51-83dd-ec9f83c58e1d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:14.313675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd9212-a6e3-4f51-83dd-ec9f83c58e1d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:14.313675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd9212-a6e3-4f51-83dd-ec9f83c58e1d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:14.315921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:36:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:14.321868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e51e5-d22a-43a9-988e-a157c63f0cc1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:14.322573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e51e5-d22a-43a9-988e-a157c63f0cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:14.322717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e51e5-d22a-43a9-988e-a157c63f0cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:14.346694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-833e51e5-d22a-43a9-988e-a157c63f0cc1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:14.347885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-833e51e5-d22a-43a9-988e-a157c63f0cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:14.357025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:14.525378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd9212-a6e3-4f51-83dd-ec9f83c58e1d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:14.541961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd9212-a6e3-4f51-83dd-ec9f83c58e1d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:14.547767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:14.548421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:14.548700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:14.548969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:14.557467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:14.559097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab9949-668c-43ac-af52-734de9e09099 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:14.568850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 764/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:15.431918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b75a7-7e75-496d-aced-aac1b887188c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:15.432556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b75a7-7e75-496d-aced-aac1b887188c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:15.432556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b75a7-7e75-496d-aced-aac1b887188c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:15.433867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:36:15.439842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e381bb-5987-4ef5-af92-d62de632b165 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:15.440618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e381bb-5987-4ef5-af92-d62de632b165 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:15.440795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e381bb-5987-4ef5-af92-d62de632b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:15.460454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-52e381bb-5987-4ef5-af92-d62de632b165 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:15.461660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52e381bb-5987-4ef5-af92-d62de632b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:15.469769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:15.621965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b75a7-7e75-496d-aced-aac1b887188c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:15.645333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b75a7-7e75-496d-aced-aac1b887188c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:15.653583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:15.654517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:15.654812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:15.655109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:15.665222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:15.666880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff4bed-89c9-4f1c-8d3c-661f75a49891 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:15.678594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:19.684999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:19.685808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:19.685808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:19.687933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:36:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:19.695403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362a602-a3bf-49ad-bafc-c371bd54e00f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:19.696111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362a602-a3bf-49ad-bafc-c371bd54e00f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:19.696219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362a602-a3bf-49ad-bafc-c371bd54e00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:19.717614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0362a602-a3bf-49ad-bafc-c371bd54e00f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:19.718809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0362a602-a3bf-49ad-bafc-c371bd54e00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:19.728572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:19.767550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:19.770163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:19.837575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77f21d6-1614-46b1-8ed5-8ce3ab1f7842 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:19.841640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:19.842289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:19.842567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:19.842841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:19.852341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:19.853788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3113f92-7d0a-4b16-9d0f-aae21c7ae522 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:19.863290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 767/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:22.077803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c6c8-baf1-478d-ba4f-100a81f01937 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:22.078426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c6c8-baf1-478d-ba4f-100a81f01937 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:22.078426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438c6c8-baf1-478d-ba4f-100a81f01937 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:22.080558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:22.089426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a6fe-88aa-44b8-93cc-4976984b3bc4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:22.089899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a6fe-88aa-44b8-93cc-4976984b3bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:22.090003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4a6fe-88aa-44b8-93cc-4976984b3bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:22.110201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9cb4a6fe-88aa-44b8-93cc-4976984b3bc4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:22.111385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9cb4a6fe-88aa-44b8-93cc-4976984b3bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:22.119610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:22.159340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438c6c8-baf1-478d-ba4f-100a81f01937 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:22.174273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438c6c8-baf1-478d-ba4f-100a81f01937 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:22.178339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:22.178980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:22.179256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:22.179523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:22.188804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:22.190274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae47131-4711-490e-ba0a-d1f17b88725a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:22.200489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:25.850864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:25.851450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:25.851450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:25.852874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:25.859445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ec0-7d0e-4331-94c6-e4f56a738e38 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:25.860191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ec0-7d0e-4331-94c6-e4f56a738e38 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:25.860379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c82ec0-7d0e-4331-94c6-e4f56a738e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:25.887792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-29c82ec0-7d0e-4331-94c6-e4f56a738e38 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:25.889017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29c82ec0-7d0e-4331-94c6-e4f56a738e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:25.897872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:26.180837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:26.183159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:26.245578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7498bc2-2b5f-4007-bc9a-b902a7242cb6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:26.250408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:26.251054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:26.251337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:26.251618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:26.261335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:26.262774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f54f97-32ae-4e37-8d4a-043bf733ff99 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:26.267752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:26.272838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 770/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:28.480004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:28.480542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:28.480542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:28.481985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:36:28.488329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2efb0-8808-46b3-858e-815c6d1c0d20 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:28.488790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2efb0-8808-46b3-858e-815c6d1c0d20 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:28.488896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2efb0-8808-46b3-858e-815c6d1c0d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:28.511662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57b2efb0-8808-46b3-858e-815c6d1c0d20 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:36:28.512883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57b2efb0-8808-46b3-858e-815c6d1c0d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:28.521808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:36:28.791566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:28.794605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:28.816200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761be33-8f7e-43b1-8ab8-c75c71626c29 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:28.821435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:28.822078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:28.822357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:28.822630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:28.831733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:28.833211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a998e99-f721-4874-ae13-057ff3fa7edd service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:28.844721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:38.504066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:38.840893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:37:58.681728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.682617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.682617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.715480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:58.716744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:58.728210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 772/1540] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:58.739151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ab7ba-cc13-4d66-9f40-0572df2633d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.740500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ab7ba-cc13-4d66-9f40-0572df2633d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.741028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ab7ba-cc13-4d66-9f40-0572df2633d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.767987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d06ab7ba-cc13-4d66-9f40-0572df2633d6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:58.769137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d06ab7ba-cc13-4d66-9f40-0572df2633d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:58.786952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1541] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:58.791556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:58.805552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ab051-26d5-43f1-9aca-3a7104ceaad4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:58.814269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:58.814958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.815239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.815508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.816277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:58.816585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:58.818439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:58.840057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 773/1542] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:58.846481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fcb60-e860-4d0a-a473-1dda49a5f31c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.846764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fcb60-e860-4d0a-a473-1dda49a5f31c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.847037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fcb60-e860-4d0a-a473-1dda49a5f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.852193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:58.866265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153b0c0-53a0-4401-b4cd-02c7b5527fd9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:58.876738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e97fcb60-e860-4d0a-a473-1dda49a5f31c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:58.878144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e97fcb60-e860-4d0a-a473-1dda49a5f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:58.880900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:58.881583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.881858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.882129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.882917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:58.883903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:37:58.883903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:37:58.883903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:37:58.885444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:58.901245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1543] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:58.914024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f702b5d-d724-4318-a197-84960c78410d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:58.914394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f702b5d-d724-4318-a197-84960c78410d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:58.914571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f702b5d-d724-4318-a197-84960c78410d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:58.948384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6f702b5d-d724-4318-a197-84960c78410d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:58.948832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 774/1544] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:37:58.950031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6f702b5d-d724-4318-a197-84960c78410d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:58.961180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.028849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1545] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:37:59.030128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3341-eb4c-4df6-8dd0-59d8faec841f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.030589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3341-eb4c-4df6-8dd0-59d8faec841f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.030867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed3341-eb4c-4df6-8dd0-59d8faec841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.056636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84ed3341-eb4c-4df6-8dd0-59d8faec841f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.058106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84ed3341-eb4c-4df6-8dd0-59d8faec841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.066527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1546] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.074948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.075341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.075636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.103564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.107118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.110182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c65b59-aba7-4b40-aeda-8bee1f35abf3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.119591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.120352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.120563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.120842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.121538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.123725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4c734-c691-410b-a11f-f43086167bb1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.129621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1547] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.130688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.141107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb5fab-3c4d-4704-9051-5b634ba8e8df None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.154977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.155715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.155994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.156294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.157024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.157328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.158439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:37:59.158439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:37:59.158439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:37:59.159322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512dc799-449b-4b99-b37c-9c46c0fc2208 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.162632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 775/1548] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:37:58 2022] POST /identity/v3/users => generated 302 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.166902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018e49e-3382-4874-b7de-f7a3df1783d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.167266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018e49e-3382-4874-b7de-f7a3df1783d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.167548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018e49e-3382-4874-b7de-f7a3df1783d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.179214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1549] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.179214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dee49-ff9a-490f-bb06-e13e534ab72f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.179214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dee49-ff9a-490f-bb06-e13e534ab72f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.179214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dee49-ff9a-490f-bb06-e13e534ab72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.185704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8018e49e-3382-4874-b7de-f7a3df1783d2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.186921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8018e49e-3382-4874-b7de-f7a3df1783d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.194926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 776/1550] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.196334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33425d3-1294-448c-9c37-bd7ce0f4e6c2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.196619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33425d3-1294-448c-9c37-bd7ce0f4e6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.196896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33425d3-1294-448c-9c37-bd7ce0f4e6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.197008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-267dee49-ff9a-490f-bb06-e13e534ab72f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.198136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-267dee49-ff9a-490f-bb06-e13e534ab72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.208816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1551] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.208816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bbbdd-56eb-4e5c-a66f-9b0222f9b1d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.208816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bbbdd-56eb-4e5c-a66f-9b0222f9b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.208816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bbbdd-56eb-4e5c-a66f-9b0222f9b1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.215609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d33425d3-1294-448c-9c37-bd7ce0f4e6c2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.216842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d33425d3-1294-448c-9c37-bd7ce0f4e6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.224785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 777/1552] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.226105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.226236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4f7bbbdd-56eb-4e5c-a66f-9b0222f9b1d9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.226559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.226953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.227262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4f7bbbdd-56eb-4e5c-a66f-9b0222f9b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.235209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1553] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.236627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d319-c582-4261-8879-0a967b04e991 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.236912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d319-c582-4261-8879-0a967b04e991 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.237157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515d319-c582-4261-8879-0a967b04e991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.245373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.246550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.254356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 778/1554] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.254810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9515d319-c582-4261-8879-0a967b04e991 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.255231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.256164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9515d319-c582-4261-8879-0a967b04e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.264184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1555] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.264499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcb7ac-7f30-4cf4-b11a-97b3e3391338 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.265017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515d319-c582-4261-8879-0a967b04e991 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.269077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.269740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.270013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.270281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.271067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.272722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cacb2f8-37c3-4e7c-b60b-d3750f6f4331 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.275161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515d319-c582-4261-8879-0a967b04e991 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.280005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.280643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.280923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.281203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.281892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.283564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 779/1556] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:37:59.283902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.284865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7a1d-56bc-4d0c-8bc6-81a1229d2e0e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.285153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7a1d-56bc-4d0c-8bc6-81a1229d2e0e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.285429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b7a1d-56bc-4d0c-8bc6-81a1229d2e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.303507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b4b7a1d-56bc-4d0c-8bc6-81a1229d2e0e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.304882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b4b7a1d-56bc-4d0c-8bc6-81a1229d2e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.311606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1557] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.312603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.317343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 780/1558] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.322112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b657a19-b06a-4c20-a84d-20725dcf4666 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.322394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b657a19-b06a-4c20-a84d-20725dcf4666 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.322664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b657a19-b06a-4c20-a84d-20725dcf4666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.340692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b657a19-b06a-4c20-a84d-20725dcf4666 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.341884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b657a19-b06a-4c20-a84d-20725dcf4666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.376250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a763b51-63d4-4c93-a6f7-c4b092c57e10 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.381602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.382267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.382865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.383162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.383896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.384537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.386155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.398059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1559] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.402158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 781/1560] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.405182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.405554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639173b6-1633-4d9d-8df5-e4c3f1f0c7c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.405881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639173b6-1633-4d9d-8df5-e4c3f1f0c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.406158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639173b6-1633-4d9d-8df5-e4c3f1f0c7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.416292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bad333-40f7-41c6-94cb-eec714d0d6d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.420690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.421337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.421613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.421883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.422594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.422896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.423871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-639173b6-1633-4d9d-8df5-e4c3f1f0c7c8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.424531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.425106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-639173b6-1633-4d9d-8df5-e4c3f1f0c7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.433128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:37:59.433343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 782/1561] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.435442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.435918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.436336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.437541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1562] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.438201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.444676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:37:59.449670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0827d1c-84c4-4244-8aaa-f56e8eedfc9a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.454504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.455167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.455553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.455824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.456231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:37:59.456659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.457435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:37:59.458430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.465695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 783/1563] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:37:59.469468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.469570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1564] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:37:59.469570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.479000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590406c-ad74-4139-b876-564a1b93f9ee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.479695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c8f98-f24f-4b0d-a3c6-6c465bb1c82c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.483545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.484215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.484493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.484651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.484778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.485462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.485569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.485759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.486004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.486365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.487079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.487349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.489055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.506331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1565] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.506331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.511183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78730bce-dc40-43ce-9e09-ef617adb5432 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.516353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.516998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.517278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.517546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.518239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.519230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 784/1566] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.520151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.520268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.530175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc0d0d-7c70-40ba-b8fc-cd26aaa2dd7e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.534890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.535536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.535844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.536134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.537124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.537424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.539220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.547151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1567] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.551247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.553272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 785/1568] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.554134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.564590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb17b4-6249-40fa-8d7d-fc2e3032634e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.612876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0975ab31-b0d2-41ed-8853-8794b8d0437d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.617616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.618268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.618565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.618832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.619549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.619879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.621503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.622274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.622903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.623182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.623481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.624202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.626104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.640058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1569] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.640058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.651397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8fd1d-86f1-4aa9-8980-8da7b36c0e9d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.656138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 786/1570] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.656138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.718563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.719233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.719441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5f47-b770-4a5a-abe5-a6c501a3c3e2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.719915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.719915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.720509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:37:59.720813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.722429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.725854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.726763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.727045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.727318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] PATH_INFO: `/v3/projects/fec78be42a884892b793a3cad52e0e26/users/5e474605e2cb4ee78b716e97dd8f0388/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.735480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1571] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:37:59.743185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.743754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec78be42a884892b793a3cad52e0e26, user_id=5e474605e2cb4ee78b716e97dd8f0388, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.744748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:37:59.744748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:37:59.744748 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:37:59.746809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.747786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ababeac-6378-4efa-afc6-2795e0d20da4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.752790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.753500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.753757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.754030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.754858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.757095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.760920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 787/1572] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:37:59 2022] PUT /identity/v3/projects/fec78be42a884892b793a3cad52e0e26/users/5e474605e2cb4ee78b716e97dd8f0388/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:37:59.761744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.792223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1573] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.792223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.805266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:37:59.811928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:37:59.830205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea868c6-9dac-40dd-b86c-46ec584588f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.855867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374e70a0-2689-4f69-976e-808575296f88 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.861207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.861878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.862153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.862408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.863110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.864172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:37:59.864172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:37:59.864172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:37:59.865522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.889162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1574] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.889162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.890893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.891641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.891917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.892216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.892963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.894550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.896803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23ef60-7d9b-4938-ad68-072b9013e977 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.904554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.905315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.908766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.908928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 788/1575] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:37:59.909017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.909104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.909188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.909188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.915799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268a6e9-29e2-4309-bde2-98aa2796dc08 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:37:59.920605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:37:59.921252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:37:59.921528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:37:59.921794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:37:59.922470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:37:59.924400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:37:59.939514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1576] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.939514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:37:59.955049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 789/1577] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:37:59.955997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.003975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec373de-8904-4b54-a49b-026218d4c778 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.026287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2aef5-5f22-4dea-8cdd-176bd4f5c838 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.031293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.031953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.032264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.032536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.033232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.035196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f790-bd54-433a-b0b9-e311bb4dd9f4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.060991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.061594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.061877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.062141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 790/1578] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.062255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.062894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.063411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.063701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.063964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.064868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.085553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1579] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:37:59 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.086344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.091470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] MFA Rules not processed for user `5e474605e2cb4ee78b716e97dd8f0388`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:00.092691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:00.095700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a27ca-dc8c-4ba0-b48d-1f7b33388d17 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.116346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.122228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.154808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.155487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.155764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.156071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.156748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.158894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.159283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 791/1580] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:00.162962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.170895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020fcaa-f293-4345-9954-49f2754f117f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.176374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.177069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.177350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.177804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.178590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.180409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1581] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.180540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.181204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.192059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 792/1582] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.192059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.195512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04189b-b9d3-419c-9339-8f5618b0320e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.200297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.200460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaaa23-a14e-4358-8514-ff69a9ce358d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.200973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.201251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.201521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.202243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.203848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.205201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.205848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.206111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.206378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.207049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.209265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.221768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1583] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.221768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.230186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cbebca-fdf6-452f-9349-2cfbb0bb35b3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.233915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 793/1584] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/users => generated 302 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.233915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.237426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.238087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.238360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.238627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.239311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.241024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fa967-09c7-47c6-b633-de5e8e1f3f84 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.241402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.245486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.246122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.246399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.246668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.247344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.249264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.262851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1585] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.263795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.272736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 794/1586] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.273173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffaaa7-a67e-44d4-a443-67d3ac7d0b26 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.273528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.278496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.279254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.279533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.279796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.280548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.283186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.284305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d01fb-bdf3-4c92-ab5a-a84046856eaa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.291608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.292315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.292599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.292881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.293630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.296339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.304143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1587] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.305112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.319836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20f81-d18a-48f6-aa08-ac20ada75920 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.324590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.325240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.325512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.325782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.326482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 795/1588] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/projects => generated 405 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.326619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.328187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.330439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.338707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1589] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.339395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.348540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295dc01-b126-4ddc-8eff-de7b89cc3e68 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.392419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e2dee6-6746-4260-a0b4-e2acf5070c06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.397210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.397852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.398127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.398397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.399118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.400819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.405828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.407170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.407426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.407693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] PATH_INFO: `/v3/projects/f86f8a870f9146489a3ef1716094e066/users/e9f6a35cac50431d9468b51c9259bdb5/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.410783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 796/1590] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.411634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.420893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8b78d-d975-405a-b7b0-6374cc0afee3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.422290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86f8a870f9146489a3ef1716094e066, user_id=e9f6a35cac50431d9468b51c9259bdb5, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.424978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.426650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.427617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.427893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.428199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.428923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.430531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.439386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1591] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/f86f8a870f9146489a3ef1716094e066/users/e9f6a35cac50431d9468b51c9259bdb5/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:00.440892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.443425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 797/1592] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.444833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.456177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31938e-d0db-4503-a8b1-4c3f1c5f303d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.488384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.501861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.504296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9d510-5295-47e7-a95a-2d6f6baa6f7e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.510137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.510942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.511280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.511497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.512256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.513869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.514982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.515683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.515952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.516578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.516978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.518586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.524855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1593] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.525972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.532167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 798/1594] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.532167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.537930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90da85b-0b46-4176-9c0d-87d5bd8a5a33 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.539274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cd67a-82b9-4af5-b92d-4f24c66991d2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.542858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.543519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.543799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.543895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.544221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.544559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.544838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.544941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.545112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.545792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.546592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.547678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a73080-9423-4478-85e7-d1cdcfad9444 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.557145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1595] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.557966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.568445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0bca3-4206-4219-b82a-38124028b3aa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.573010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.573588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.573866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.574137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.574868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.576533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.586721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1596] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.587467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.596691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9284c6-b7e6-4a48-8863-a263938c6e6b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.601386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.602259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.602538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.602811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] PATH_INFO: `/v3/projects/ad0aa5dae88a4adfad862d64222f17ae/users/953fe7550e4349f6ae22896621590ceb/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.616354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 799/1597] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/users => generated 309 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:00.617832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.618122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.618393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.620634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0aa5dae88a4adfad862d64222f17ae, user_id=953fe7550e4349f6ae22896621590ceb, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.623476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.635632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:00.636877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:00.645237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 800/1598] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:00.646243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.648894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1599] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/ad0aa5dae88a4adfad862d64222f17ae/users/953fe7550e4349f6ae22896621590ceb/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:00.658808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.658929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee512e4f-9df6-4d0d-a5c4-951cdba69ee9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.720905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.721585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.721859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.722130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.722295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128915f-d604-41f1-a112-8f7b0ed2d8e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.722962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.724644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.727053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.727775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.728307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.728606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.729333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.730939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.738175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 801/1600] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.738175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.742286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1601] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.742986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.746577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fac512-7ed1-4d37-b4ec-2f623f84903f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.751950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.752332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfa5ab-ea77-4132-8d55-ac538fb9d36a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.752786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.752882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.753141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.753883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.755534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.756932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.757837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.758087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.758360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] PATH_INFO: `/v3/projects/e280d545f6064b5c91289a77e2491511/users/6d07884e92bc4cfe95d2fd99c40f25f0/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.765947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 802/1602] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.766792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.773961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e280d545f6064b5c91289a77e2491511, user_id=6d07884e92bc4cfe95d2fd99c40f25f0, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.776673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.776853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316fae5f-57cc-4e3f-9131-0dd35d31f4ed admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.782153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.783033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.783338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.783581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] PATH_INFO: `/v3/projects/1dced399c740478d9282fe6f2bf05a7d/users/b74d37f9e16a417baf03a8bbea2b5adf/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.790760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1603] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/e280d545f6064b5c91289a77e2491511/users/6d07884e92bc4cfe95d2fd99c40f25f0/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:00.791839 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.801334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dced399c740478d9282fe6f2bf05a7d, user_id=b74d37f9e16a417baf03a8bbea2b5adf, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.804322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe49a4-bafe-4b7f-9690-9f0998c7dbef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.819006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 803/1604] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/1dced399c740478d9282fe6f2bf05a7d/users/b74d37f9e16a417baf03a8bbea2b5adf/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:00.825480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.825700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.825969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.854803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] MFA Rules not processed for user `953fe7550e4349f6ae22896621590ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:00.856014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:00.863144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec2769-f6c6-4139-a40e-0611d201f204 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.875147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.920788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.921462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.921727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.921859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 804/1605] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:00.922045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.922699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.922802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.924201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:00.924425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.934711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1606] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:00.935594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.941064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:00.946253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6ab9-469e-4314-bcde-5be5e74be750 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:00.951141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:00.952051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:00.952348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:00.952621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] PATH_INFO: `/v3/projects/114e4162925c474bb7d49d7d0dbf11e8/users/2e9f5aabc2be448cba49802ddbef2e96/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:00.967670 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=114e4162925c474bb7d49d7d0dbf11e8, user_id=2e9f5aabc2be448cba49802ddbef2e96, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:00.970390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:00.984297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1607] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/114e4162925c474bb7d49d7d0dbf11e8/users/2e9f5aabc2be448cba49802ddbef2e96/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:00.985095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:00.990810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f3cb-12f9-4cd7-a315-71d1af292cae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.040164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.040816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.041123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.041396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.046063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e38f376-4912-4403-993e-73c63edd8c29 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.050470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.051113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.052045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.052176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.052364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.052632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] PATH_INFO: `/v3/projects/2a6ea8ff8d114436bca2e300d9c01eea/users/eff6a47a57734cc5828b6187d4cc908f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.061847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 805/1608] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:01.062722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.069039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6ea8ff8d114436bca2e300d9c01eea, user_id=eff6a47a57734cc5828b6187d4cc908f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.071837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241cf8-e60d-4770-9f18-34ebbc25fbeb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.071954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0d42f-f7d0-4b07-b990-b71590072b39 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.080952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.081897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.082161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.082484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] PATH_INFO: `/v3/projects/fd0613f4cbb3461a982738c683c3ab03/users/9bb20bf4c31d44ac9dd9ef0db9a2a63a/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.090105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1609] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:00 2022] PUT /identity/v3/projects/2a6ea8ff8d114436bca2e300d9c01eea/users/eff6a47a57734cc5828b6187d4cc908f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:01.098588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.098942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.099780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.101474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0613f4cbb3461a982738c683c3ab03, user_id=9bb20bf4c31d44ac9dd9ef0db9a2a63a, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.104620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4b3cf3-b874-4b6d-8018-01757232d0c1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.119092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.120401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.121681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 806/1610] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:01 2022] PUT /identity/v3/projects/fd0613f4cbb3461a982738c683c3ab03/users/9bb20bf4c31d44ac9dd9ef0db9a2a63a/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:01.127112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.127636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.128002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.161169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] MFA Rules not processed for user `b74d37f9e16a417baf03a8bbea2b5adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.162835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.186756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1611] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.187616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.198185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b549ce-00ce-467a-b9f5-9dcd8f71d130 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.203608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.204449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.204733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.205027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.205779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.207760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53234745-dd27-4c04-813f-5bd9775bb12d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.216980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:01.239518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 807/1612] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.240434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.242557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1613] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:01.244016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b111c1-4734-4211-a064-7b0596733dbe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.244332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b111c1-4734-4211-a064-7b0596733dbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.244612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b111c1-4734-4211-a064-7b0596733dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.255486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8318c-d754-4988-b403-47633cc9da13 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.267857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-77b111c1-4734-4211-a064-7b0596733dbe None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.267857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77b111c1-4734-4211-a064-7b0596733dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.301964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:01.335194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1614] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.335766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.336521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.336812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.337087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.337350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.337764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.338252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] PATH_INFO: `/v3/projects/7fc9c6167079494784ea25cc4b894269/users/f0604310a2974bb0bcaeb34dd5d7416e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.357553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc9c6167079494784ea25cc4b894269, user_id=f0604310a2974bb0bcaeb34dd5d7416e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.360332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478f68c6-5eec-4292-8dd4-299c23bbf1ec admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.368818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] MFA Rules not processed for user `eff6a47a57734cc5828b6187d4cc908f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.370965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.374780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 808/1615] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:01 2022] PUT /identity/v3/projects/7fc9c6167079494784ea25cc4b894269/users/f0604310a2974bb0bcaeb34dd5d7416e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:01.376456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.376744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.377785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.395218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.397200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.469223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 809/1616] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.470491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.484631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1617] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.485471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.499102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd89f61-e4ee-462d-a883-7d0975ee2cad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.508629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.509296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.509828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.510110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.511209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.513372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.542769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1618] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/users => generated 296 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:01.543659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.553822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893e10e-7668-4c6b-8689-84c85c167f6a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.559545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.560338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.560620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.560897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.561598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.563705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.567263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72ed4e-6322-40ea-bb46-9471f3a5ab26 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.571177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.572059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.572327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.572594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.582839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.584854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.596043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1619] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:01.596043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.596777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 810/1620] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:01.597249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.608888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26f585-085b-4147-8001-2976e57bb6a4 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.644364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:01.660882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3e0f2d-7c62-4a52-a790-c250e00cb25f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.670214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.671271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.671709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.672186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.682268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.682955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.683234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.683505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.684356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.685873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.690226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:01.718293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 811/1621] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:01.718293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.731205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d76e19-a1c9-46ec-a53b-38e24c750147 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.735720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.736263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.736538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.736813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.748600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.750049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29038c66-80f2-439e-b7f1-583f9487d66f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.759450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.761506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.762315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 812/1622] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:01.763988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e37e-4671-4351-a305-c96903d42e96 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.764300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e37e-4671-4351-a305-c96903d42e96 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.764582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e37e-4671-4351-a305-c96903d42e96 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.766365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 813/1623] 10.222.0.55 () {54 vars in 961 bytes} [Tue May 24 15:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:01.766611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86e37e-4671-4351-a305-c96903d42e96 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.775902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1624] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:01.776627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.782220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86e37e-4671-4351-a305-c96903d42e96 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.788289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.788598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14edeb6-7402-428b-b11d-eddec3145474 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.789293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.789577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.789848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.790625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.793001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.795286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.796244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.796592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.797004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.797801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.799933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c8f56-9c5c-432b-b6a2-e24b75b4a32a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.819633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 814/1625] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:01.820172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.833088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1626] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:01.834814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.835126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.835449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.850653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b2455-71a0-40d8-ab4a-5d787ecf7060 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:01.862251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:01.863793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:01.872896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:01.873833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:01.874106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:01.874382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] PATH_INFO: `/v3/projects/7fc9c6167079494784ea25cc4b894269/users/f0604310a2974bb0bcaeb34dd5d7416e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:01.876460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc9c6167079494784ea25cc4b894269, user_id=f0604310a2974bb0bcaeb34dd5d7416e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:01.879131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:01.914806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 815/1627] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:01 2022] PUT /identity/v3/projects/7fc9c6167079494784ea25cc4b894269/users/f0604310a2974bb0bcaeb34dd5d7416e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:01.914806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.965749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1628] 10.222.0.55 () {58 vars in 1058 bytes} [Tue May 24 15:38:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:01.966646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:01.977860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14767ba9-8887-414d-b808-283e748d0f47 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.023035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db353cd-a6aa-4974-926a-0867046c7626 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.034896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.035823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.036180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.036500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] PATH_INFO: `/v3/projects/f1b0c43108184906ba9f23c021f0345a/users/db24427e62444d8e8380e9f779da4eeb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.061005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.061737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.061935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.062213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.062944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.064597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f8878a-de0e-4c64-8f0b-30de1706e126 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.066813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b0c43108184906ba9f23c021f0345a, user_id=db24427e62444d8e8380e9f779da4eeb, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.069806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe5fc63-518a-4eb8-92cb-b2200b6f049e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.077327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:02.090734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 816/1629] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:01 2022] PUT /identity/v3/projects/f1b0c43108184906ba9f23c021f0345a/users/db24427e62444d8e8380e9f779da4eeb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:02.095247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1630] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:01 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:02.096827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.097117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.097393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.099698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.099988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.100407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.151492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] MFA Rules not processed for user `e9f6a35cac50431d9468b51c9259bdb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:02.153687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] MFA Rules not processed for user `f0604310a2974bb0bcaeb34dd5d7416e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:02.157150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:02.159269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:02.255591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 817/1631] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:02.257042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.271723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1632] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:02.271723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.334825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:02.366568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200230d-3356-4fdf-abfc-760dcef69af9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.374544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.376061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.376273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.376553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.378072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.380353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.398306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1633] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:02.400978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.401910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f5011-fe74-41fd-8dbd-727cd725ec55 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.413110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.413794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.414079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.414350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.415292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdee25b-cfb3-4e39-8cef-0fe7fa52a0cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.422342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.424229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.425158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.425771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] PATH_INFO: `/v3/projects/eaf20b463fd842f8819bf70f3280ec4c/users/570e486f8b01428296fcf8da7b993a80/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.445447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf20b463fd842f8819bf70f3280ec4c, user_id=570e486f8b01428296fcf8da7b993a80, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.449002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0c3e-1aaa-4272-ac68-e7cffe815531 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.477309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1634] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:02 2022] PUT /identity/v3/projects/eaf20b463fd842f8819bf70f3280ec4c/users/570e486f8b01428296fcf8da7b993a80/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:02.477309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.477309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.477309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.493307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.495224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.500522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] MFA Rules not processed for user `e9f6a35cac50431d9468b51c9259bdb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:02.502070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:02.511717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 818/1635] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:02.512611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.589945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1636] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:02.589945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.611293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:02.616957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df823b-d2af-41a0-9ba5-0d574c753114 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.739760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43bbfa4-8670-4928-821b-bc51d48fe2f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.741428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.742142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.742424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.743070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.750497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.751777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.752414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.752897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] PATH_INFO: `/v3/projects/f1b0c43108184906ba9f23c021f0345a/users/db24427e62444d8e8380e9f779da4eeb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.755373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b0c43108184906ba9f23c021f0345a, user_id=db24427e62444d8e8380e9f779da4eeb, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.762630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.779243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1637] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:02 2022] PUT /identity/v3/projects/f1b0c43108184906ba9f23c021f0345a/users/db24427e62444d8e8380e9f779da4eeb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:02.780691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:02.793448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:02.819183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.820791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e7fc0-e7b7-448a-ad4c-e9a36165ea1b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.834317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 819/1638] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:02 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:02.835937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a135-825b-4009-a1af-8b1be58f257d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.836280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a135-825b-4009-a1af-8b1be58f257d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.836566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a135-825b-4009-a1af-8b1be58f257d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.838737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 820/1639] 10.222.0.55 () {54 vars in 961 bytes} [Tue May 24 15:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:02.845043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.845326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.845604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.882249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] MFA Rules not processed for user `db24427e62444d8e8380e9f779da4eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:02.883379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:02.963386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef9d28-1b19-44ac-9649-e1bda8bdea83 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:02.969001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:02.969785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.970161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.970532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:02.971411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:02.974293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cece6e5-5e95-45e9-9ab1-62b219f626ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:02.992698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1640] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:02 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:02.994641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6fd8-682b-4b1b-8307-96810953a2d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:02.995087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6fd8-682b-4b1b-8307-96810953a2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:02.995462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6fd8-682b-4b1b-8307-96810953a2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.012247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 821/1641] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:03.013101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.020389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ebec6fd8-682b-4b1b-8307-96810953a2d0 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:03.022056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ebec6fd8-682b-4b1b-8307-96810953a2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:03.024641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2598c8c6-38bd-453a-be5e-bcc36356a20f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:03.035295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:03.036608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.036608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.036892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] PATH_INFO: `/v3/projects/eaf20b463fd842f8819bf70f3280ec4c/users/570e486f8b01428296fcf8da7b993a80/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.039618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaf20b463fd842f8819bf70f3280ec4c, user_id=570e486f8b01428296fcf8da7b993a80, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:03.043374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:03.056091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 822/1642] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:03 2022] PUT /identity/v3/projects/eaf20b463fd842f8819bf70f3280ec4c/users/570e486f8b01428296fcf8da7b993a80/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:03.057539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.182275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1643] 10.222.0.55 () {58 vars in 1058 bytes} [Tue May 24 15:38:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:03.184491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e67c0-c6a7-4580-9eb6-bda709b6e5af None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.184780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e67c0-c6a7-4580-9eb6-bda709b6e5af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.185538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e67c0-c6a7-4580-9eb6-bda709b6e5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.212144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8cfaf-c111-4ef6-8854-3ce5dc2f04da admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:03.217669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:03.218580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.218934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.219429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.226931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-823e67c0-c6a7-4580-9eb6-bda709b6e5af None None] MFA Rules not processed for user `570e486f8b01428296fcf8da7b993a80`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:03.228363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-823e67c0-c6a7-4580-9eb6-bda709b6e5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:03.245906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:03.365925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:03.367412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1644] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:03.368676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959b-cb70-45ce-8dbc-a117e6641c95 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.368964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959b-cb70-45ce-8dbc-a117e6641c95 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.369238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959b-cb70-45ce-8dbc-a117e6641c95 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.371120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:03.377680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:03.392193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1645] 10.222.0.55 () {54 vars in 912 bytes} [Tue May 24 15:38:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:03.394019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951959b-cb70-45ce-8dbc-a117e6641c95 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.405588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 823/1646] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:03.407216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.555110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951959b-cb70-45ce-8dbc-a117e6641c95 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:03.565683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:03.566516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.566967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.567243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.586295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb98d7-95ba-4cca-848c-2ea995e7d190 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:03.605941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:03.607929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.609367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.609988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] PATH_INFO: `/v3/users/5e474605e2cb4ee78b716e97dd8f0388` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.611850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e474605e2cb4ee78b716e97dd8f0388)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:03.616909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:38:03.616909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: warnings.warn(msg) May 24 15:38:03.616909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: May 24 15:38:03.624755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:03.756550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:03.759752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c02561-321f-4d9c-ba61-ae5472400bb7 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:03.769601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e474605e2cb4ee78b716e97dd8f0388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:03.782591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1647] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:03.795137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e474605e2cb4ee78b716e97dd8f0388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:03.797994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e474605e2cb4ee78b716e97dd8f0388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:03.798159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44206-75f9-406c-8974-225ba0393450 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.798473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44206-75f9-406c-8974-225ba0393450 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.800261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e44206-75f9-406c-8974-225ba0393450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.828684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e474605e2cb4ee78b716e97dd8f0388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:03.832117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ba4664-cc5e-4eeb-bfca-a4fb6a7d7e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e474605e2cb4ee78b716e97dd8f0388'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:03.844586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-26e44206-75f9-406c-8974-225ba0393450 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:03.846091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26e44206-75f9-406c-8974-225ba0393450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:03.865949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1648] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:03.867037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44206-75f9-406c-8974-225ba0393450 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.873759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26e44206-75f9-406c-8974-225ba0393450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:03.893463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e44206-75f9-406c-8974-225ba0393450 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:03.905415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:03.906100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.906385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.906664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.908577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:03.931360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 824/1649] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:03 2022] DELETE /identity/v3/users/5e474605e2cb4ee78b716e97dd8f0388 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:03.935084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24d32b-9733-410f-a8a8-397a8d4aa647 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:03.935370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24d32b-9733-410f-a8a8-397a8d4aa647 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:03.935597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:03.935738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24d32b-9733-410f-a8a8-397a8d4aa647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:03.937145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:03.968265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1650] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:03.970710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:03.976697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.001334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476df8-49d8-4cac-b728-fbe8ae433f4f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.043693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c24d32b-9733-410f-a8a8-397a8d4aa647 None None] MFA Rules not processed for user `2e9f5aabc2be448cba49802ddbef2e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:04.045618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c24d32b-9733-410f-a8a8-397a8d4aa647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.082356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:04.137978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.138649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.138930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.139231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.144506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.202284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 825/1651] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:04.204414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0577-990a-428a-9f72-4325af04638b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.204757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0577-990a-428a-9f72-4325af04638b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.205164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb0577-990a-428a-9f72-4325af04638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.225010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:04.239917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-61bb0577-990a-428a-9f72-4325af04638b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:04.241738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-61bb0577-990a-428a-9f72-4325af04638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.304378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.305907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.328431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1652] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:04.329321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.350356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 826/1653] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:04.350931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb0577-990a-428a-9f72-4325af04638b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.354402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-61bb0577-990a-428a-9f72-4325af04638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.366029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab788a9c-2b3f-436b-9e65-d9b40bfdc2df service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.367464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb0577-990a-428a-9f72-4325af04638b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.372564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.373396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.373734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.374067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.374951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.375075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.375648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.375927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.376236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.377475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.430331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 827/1654] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:04 2022] POST /identity/v3/projects => generated 403 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:04.432195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.434977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.539370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.544761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.571909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1655] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:04.574354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.589802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c52fc-f8d2-429a-82ba-bbdd056f702e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.594130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2519b9fa-f16d-499c-bfe4-100cae200579 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.599215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.601204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.601501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.602495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.756149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.758119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.758438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.759171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.760823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.769159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbae14-4119-4bd3-98f0-1d969b7ae9da admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.802249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:04.822704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1656] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:04 2022] POST /identity/v3/users => generated 308 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:04.823763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea25fa-74da-4591-a663-bfaf6cc57703 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.826284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea25fa-74da-4591-a663-bfaf6cc57703 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.826644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea25fa-74da-4591-a663-bfaf6cc57703 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.830384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1657] 10.222.0.55 () {54 vars in 912 bytes} [Tue May 24 15:38:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:04.831154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea25fa-74da-4591-a663-bfaf6cc57703 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.839702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.842778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb50b8-a7ed-45e7-81c2-ff0ea3469e57 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.860660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea25fa-74da-4591-a663-bfaf6cc57703 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:04.869046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:04.869787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.870043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.870323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.871145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:04.873005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:04.882322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 828/1658] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:04.883689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:04.883978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:04.884280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:04.906533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1659] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:04.906533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:04.927645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:04.929108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:04.997840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:05.016897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775359-387a-4a1e-a850-a8302c09f60a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:05.025821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:05.026545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:05.026830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:05.027107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:05.033194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:05.075163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 829/1660] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:38:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:05.076472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:05.111221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c79c37-0eaf-45c3-a926-e522c81402a6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:05.125731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:05.131176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:05.131805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:05.132966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] PATH_INFO: `/v3/projects/6a79610059a24cf1bd3892d27cd767b2/users/947d0184062c48b5a10a1aa741377be2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:05.166792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:05.170558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:05.183438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a79610059a24cf1bd3892d27cd767b2, user_id=947d0184062c48b5a10a1aa741377be2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:05.189266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1661] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:05.190316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:05.191963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:05.214180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:05.263764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 830/1662] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:05 2022] PUT /identity/v3/projects/6a79610059a24cf1bd3892d27cd767b2/users/947d0184062c48b5a10a1aa741377be2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:05.270341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:05.308810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f970d9-746d-4c24-9928-9823bf0e2120 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:05.397745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:05.398765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:05.399200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:05.399911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:05.412114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8a3dc-a034-4a56-8572-9e7e0a5d7c3e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:05.415741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:05.424538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:05.425231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:05.425514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:05.425797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:05.426625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:05.432708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:05.453708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 831/1663] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:05.459677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:05.474952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6d1ec-3844-4b54-b1ce-a1006093b2cf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:05.489827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:05.490941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:05.491220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:05.491495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] PATH_INFO: `/v3/projects/6a79610059a24cf1bd3892d27cd767b2/users/947d0184062c48b5a10a1aa741377be2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:05.493576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a79610059a24cf1bd3892d27cd767b2, user_id=947d0184062c48b5a10a1aa741377be2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:05.496255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c68aa-ed32-47e5-9c6a-d4c4c2771c03 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:05.531621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 832/1664] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:05 2022] PUT /identity/v3/projects/6a79610059a24cf1bd3892d27cd767b2/users/947d0184062c48b5a10a1aa741377be2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:05.644875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:05.649572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ee9b-f460-4cd0-9506-6a150cf33fa2 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:05.684879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1665] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:06.463898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:06.498756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:06.498756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:06.498756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 833/1666] 10.222.0.55 () {54 vars in 912 bytes} [Tue May 24 15:38:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:06.499677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:06.499677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:06.499677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:06.600012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:06.601478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:06.772860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1667] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:38:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:06.821125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:06.823598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:06.872246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:06.881216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:06.893014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942f10d-cc82-4060-ad81-ab7c15fad5b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:06.910384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:06.911090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:06.911090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:06.911756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:06.931019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:06.932630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:06.949139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1668] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:07.017693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910f4b9-decb-4077-b989-701e2f635442 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:07.032467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:07.033643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:07.033955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:07.034255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] PATH_INFO: `/v3/projects/fec78be42a884892b793a3cad52e0e26` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:07.036921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec78be42a884892b793a3cad52e0e26)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:07.039087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:07.182159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:07.229828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec78be42a884892b793a3cad52e0e26', 'request_id': 'req-b2986710-108f-468f-bfb2-209111d0ecad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.268303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec78be42a884892b793a3cad52e0e26', 'request_id': 'req-b2986710-108f-468f-bfb2-209111d0ecad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.285970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec78be42a884892b793a3cad52e0e26', 'request_id': 'req-b2986710-108f-468f-bfb2-209111d0ecad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.323076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] The token cache is being invalidate because project fec78be42a884892b793a3cad52e0e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:07.323375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.324866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2986710-108f-468f-bfb2-209111d0ecad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.329771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 834/1669] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:06 2022] DELETE /identity/v3/projects/fec78be42a884892b793a3cad52e0e26 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:07.330955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:07.369374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2e631-d2ea-4fa2-9ff8-9a1d26e89aaf service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:07.581043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:07.582126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:07.582566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:07.583637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] PATH_INFO: `/v3/users/f0604310a2974bb0bcaeb34dd5d7416e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:07.585287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0604310a2974bb0bcaeb34dd5d7416e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:07.588595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:07.650932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:07.721727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0604310a2974bb0bcaeb34dd5d7416e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.743116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0604310a2974bb0bcaeb34dd5d7416e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.743905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0604310a2974bb0bcaeb34dd5d7416e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.744741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0604310a2974bb0bcaeb34dd5d7416e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.763607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eec0d78-3cea-447c-8ae7-92aa5c09614b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0604310a2974bb0bcaeb34dd5d7416e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:07.797165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1670] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:07 2022] DELETE /identity/v3/users/f0604310a2974bb0bcaeb34dd5d7416e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:08.387671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:08.388241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:08.388241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:08.444280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] MFA Rules not processed for user `947d0184062c48b5a10a1aa741377be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:08.445703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:08.476219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94658c0b-be57-4e9d-a97e-068fd7aba0cc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:08.476721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94658c0b-be57-4e9d-a97e-068fd7aba0cc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:08.476820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94658c0b-be57-4e9d-a97e-068fd7aba0cc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:08.478715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1671] 10.222.0.55 () {54 vars in 960 bytes} [Tue May 24 15:38:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:08.484880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:08.485606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:08.486074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:08.544151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:08.547565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:08.594977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 835/1672] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:08.721131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1673] 10.222.0.55 () {58 vars in 1057 bytes} [Tue May 24 15:38:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:08.721716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:08.735388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d01bf-cf09-4d90-973b-8e917d1f8ac3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:08.745057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:08.748062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:08.748416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:08.748703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:08.752146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:08.769116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:08.773045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8284cf0-fef0-4077-8ec7-f2b915d0a852 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:08.796613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 836/1674] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:09.989443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:10.001045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.124143 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8773da-638f-46cc-8c99-3278567a3c19 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.124626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8773da-638f-46cc-8c99-3278567a3c19 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.124704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8773da-638f-46cc-8c99-3278567a3c19 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.127005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 837/1675] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:38:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:10.162973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.163547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.164107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.207461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32459c-cf96-4fca-8b13-97a22d82fc4f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:10.227898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:10.238747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.238939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:10.239901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.244710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.245049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] PATH_INFO: `/v3/projects/7fc9c6167079494784ea25cc4b894269` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.246565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc9c6167079494784ea25cc4b894269)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:10.248866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: WARNING py.warnings [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:38:10.248866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: warnings.warn(msg) May 24 15:38:10.248866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: May 24 15:38:10.250643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:10.279835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 838/1676] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:38:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:10.303789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:10.305231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.326001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d964781-e319-4fab-9dcb-cb2b948dfd47 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:10.343250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:10.344198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.346972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.347674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.373488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:10.375822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b315f-834a-4587-b2a5-b8202abb95d2 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:10.376455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:10.408288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 839/1677] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:10.426657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc9c6167079494784ea25cc4b894269', 'request_id': 'req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:10.442386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc9c6167079494784ea25cc4b894269', 'request_id': 'req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:10.470549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc9c6167079494784ea25cc4b894269', 'request_id': 'req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:10.508167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] The token cache is being invalidate because project 7fc9c6167079494784ea25cc4b894269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:10.508491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:10.510788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b26b5d72-9a5f-45f4-b62b-b9fd9cafcc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:10.513851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1678] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:09 2022] DELETE /identity/v3/projects/7fc9c6167079494784ea25cc4b894269 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:10.519647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.519933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.520266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.551335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:10.556612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.703207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 840/1679] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:10.715936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737113a-634e-4008-bbb5-895278f4325b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.716461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737113a-634e-4008-bbb5-895278f4325b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.716833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f737113a-634e-4008-bbb5-895278f4325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.769410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f737113a-634e-4008-bbb5-895278f4325b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:10.773636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f737113a-634e-4008-bbb5-895278f4325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.800859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1680] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:10.806116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:10.871036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721669d4-4643-4e62-ade9-babb8f4a8449 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:10.901185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:10.904147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.904805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.905930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.907010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:10.908448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:10.913327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:10.920883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:10.952087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737113a-634e-4008-bbb5-895278f4325b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:10.954723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 841/1681] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:10.961658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f737113a-634e-4008-bbb5-895278f4325b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:10.972082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:10.975194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.976493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.976891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.978282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:10.986629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.064687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1682] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:10 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:11.068580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.087906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca35444-948c-4528-aa86-53acbc81f488 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.217633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.218323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.218605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.218873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.219567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.224967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.271856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 842/1683] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:11 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:11.273680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.299634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ba2a8-3927-4c37-8027-aacbd3c49943 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.321786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.322599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.325336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.325756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.326682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.328438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.386259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1684] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:11.390333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.404304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6529c7-0fe6-49cf-ac20-d8f398956520 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.418738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.419461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.419746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.420023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.421498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.426227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.470207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 843/1685] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:11.472613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.486791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d08ef1-66ae-43cd-b0e5-5def60ee6119 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.498612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.500899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.501435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.502279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] PATH_INFO: `/v3/projects/a22d23d88c81484da4ce2699565d6b85/users/c63ef19910e1425faeec4314751a980f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.534706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.536891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:11.550014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22d23d88c81484da4ce2699565d6b85, user_id=c63ef19910e1425faeec4314751a980f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.553095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d59abd-e08b-4aa8-ab2e-aae42219e4dd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.575783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672365e7-6520-4d48-8de6-73645089779f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.592772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1686] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:11 2022] PUT /identity/v3/projects/a22d23d88c81484da4ce2699565d6b85/users/c63ef19910e1425faeec4314751a980f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:11.598471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.601556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.602000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.682848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] MFA Rules not processed for user `c63ef19910e1425faeec4314751a980f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:11.689449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:11.721524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.722692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.722969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.723240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] PATH_INFO: `/v3/users/2e9f5aabc2be448cba49802ddbef2e96` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.728973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9f5aabc2be448cba49802ddbef2e96)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.731692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.798228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1687] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:11.805464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.810029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9f5aabc2be448cba49802ddbef2e96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:11.819496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9f5aabc2be448cba49802ddbef2e96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:11.820321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9f5aabc2be448cba49802ddbef2e96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:11.831357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be589e1-4875-4084-baed-a359e54d23b7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:11.832534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9f5aabc2be448cba49802ddbef2e96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:11.833092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9f5aabc2be448cba49802ddbef2e96'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:11.841207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:11.841830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:11.842112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:11.842479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:11.843183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:11.845213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:11.849472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:11.865610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 844/1688] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:11 2022] DELETE /identity/v3/users/2e9f5aabc2be448cba49802ddbef2e96 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:11.902508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:11.902655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1689] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:11 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:11.904411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:11.992691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e76cb-bf99-4f83-83a9-3d152e463204 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.066865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.069467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.069932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.070509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.071557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.073217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.102445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.112238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 845/1690] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:11 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:12.115352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7819-0f86-4686-8f4b-7717c92c9894 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.120506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.121266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.122440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.122781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.123966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.126346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.147350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1691] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:12.153738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.171094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbd6c5-529b-4505-9e2f-ca83a3e0d356 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.176864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.177903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.178188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.178731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] PATH_INFO: `/v3/projects/b217ce25001e467f92a18edd1afa3cd0/users/be2b5b0b240d4b5f82f5d52eb01dcb1e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.199541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b217ce25001e467f92a18edd1afa3cd0, user_id=be2b5b0b240d4b5f82f5d52eb01dcb1e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.203443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.230324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.231906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 846/1692] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:12 2022] PUT /identity/v3/projects/b217ce25001e467f92a18edd1afa3cd0/users/be2b5b0b240d4b5f82f5d52eb01dcb1e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:12.244621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60ec53-5e1e-44fc-a898-a270f9dd5364 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.266351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:12.348856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.349389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.349710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.349975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.350720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.352522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879dba65-69ac-46d2-b5b5-c5ae824ad02b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.379139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1693] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:12.383786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.418596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee30010-20ad-43d0-bb74-70a2f1fdb6bb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.425464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.427494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.427794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.428101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] PATH_INFO: `/v3/projects/b217ce25001e467f92a18edd1afa3cd0/users/be2b5b0b240d4b5f82f5d52eb01dcb1e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.433465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b217ce25001e467f92a18edd1afa3cd0, user_id=be2b5b0b240d4b5f82f5d52eb01dcb1e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.437077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.465946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 847/1694] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:12 2022] PUT /identity/v3/projects/b217ce25001e467f92a18edd1afa3cd0/users/be2b5b0b240d4b5f82f5d52eb01dcb1e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:12.478819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.479189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.479789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.514435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] MFA Rules not processed for user `be2b5b0b240d4b5f82f5d52eb01dcb1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:12.515698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:12.626460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1695] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:12.640671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.643753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:12.694061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.696301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:12.779427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f1336-8046-4642-9d9e-39acb45c9a66 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.785231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.787536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.787774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.788328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.792610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331c871-c1bc-4faf-b9b2-239240ba6c59 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:12.799661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:12.799867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:12.801139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:12.801431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:12.802710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] PATH_INFO: `/v3/projects/114e4162925c474bb7d49d7d0dbf11e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:12.803614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=114e4162925c474bb7d49d7d0dbf11e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.806039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.812152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:12.816707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:12.859539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 848/1696] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:12.870367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:12.881722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '114e4162925c474bb7d49d7d0dbf11e8', 'request_id': 'req-7f2320ec-3441-440d-ae26-f242934fa2c3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:12.906694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '114e4162925c474bb7d49d7d0dbf11e8', 'request_id': 'req-7f2320ec-3441-440d-ae26-f242934fa2c3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:12.918392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '114e4162925c474bb7d49d7d0dbf11e8', 'request_id': 'req-7f2320ec-3441-440d-ae26-f242934fa2c3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:12.933687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] The token cache is being invalidate because project 114e4162925c474bb7d49d7d0dbf11e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:12.934509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:12.935396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7f2320ec-3441-440d-ae26-f242934fa2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:12.940022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1697] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:12 2022] DELETE /identity/v3/projects/114e4162925c474bb7d49d7d0dbf11e8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:12.941315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:12.954247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639b6c9-5563-4057-a06c-44e09ff80847 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:13.096439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:13.097696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.098072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.098443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] PATH_INFO: `/v3/users/947d0184062c48b5a10a1aa741377be2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.101612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947d0184062c48b5a10a1aa741377be2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:13.103732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:13.175196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb7f20-1f25-4612-b577-f1a838093b74 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.175196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb7f20-1f25-4612-b577-f1a838093b74 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.175838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb7f20-1f25-4612-b577-f1a838093b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.207008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48cb7f20-1f25-4612-b577-f1a838093b74 None None] MFA Rules not processed for user `6d07884e92bc4cfe95d2fd99c40f25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:13.208876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48cb7f20-1f25-4612-b577-f1a838093b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.228954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947d0184062c48b5a10a1aa741377be2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.250196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947d0184062c48b5a10a1aa741377be2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.252445 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947d0184062c48b5a10a1aa741377be2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.265017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947d0184062c48b5a10a1aa741377be2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.266077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-809f8c5d-4b86-4503-b85b-1ada75c983a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947d0184062c48b5a10a1aa741377be2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.311565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 849/1698] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:12 2022] DELETE /identity/v3/users/947d0184062c48b5a10a1aa741377be2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:13.379880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1699] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:13.390763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.391136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.391491 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.425054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.425347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.425974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.464778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] MFA Rules not processed for user `6d07884e92bc4cfe95d2fd99c40f25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:13.466043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.515140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] MFA Rules not processed for user `9bb20bf4c31d44ac9dd9ef0db9a2a63a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:13.522919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.690151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 850/1700] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:13.692348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:13.694120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.749563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:13.761174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1701] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:13.762037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:13.768496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.820829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b1e9a-2e7d-4653-80cd-1035d178860e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:13.826070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be2b5b0b240d4b5f82f5d52eb01dcb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b217ce25001e467f92a18edd1afa3cd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:13.827046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.827604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.828074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.829108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:13.829587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:13.831530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:13.843332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:13.847503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 851/1702] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:13.852810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:13.854799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:13.870565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa191f5-80a1-46de-a1df-df4ab6922d89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:13.877211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:13.878843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:13.879204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:13.880882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] PATH_INFO: `/v3/projects/6a79610059a24cf1bd3892d27cd767b2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:13.882623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a79610059a24cf1bd3892d27cd767b2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:13.885353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:13.965356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a79610059a24cf1bd3892d27cd767b2', 'request_id': 'req-c0896310-ff8f-47c9-80ed-e39523dc0534'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.974742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a79610059a24cf1bd3892d27cd767b2', 'request_id': 'req-c0896310-ff8f-47c9-80ed-e39523dc0534'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:13.979941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d24a58-bbc3-42f2-aa39-15514ad881ee tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:13.986349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a79610059a24cf1bd3892d27cd767b2', 'request_id': 'req-c0896310-ff8f-47c9-80ed-e39523dc0534'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:14.003557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] The token cache is being invalidate because project 6a79610059a24cf1bd3892d27cd767b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:14.004078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:14.004962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:14.006929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1703] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:13 2022] DELETE /identity/v3/projects/6a79610059a24cf1bd3892d27cd767b2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:14.007803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.010087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.097317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.099596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.100099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.101603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.103296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.136856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0896310-ff8f-47c9-80ed-e39523dc0534 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.155705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.157221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.157575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.157957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.160604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.190484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:14.207069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.209989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.225040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 852/1704] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:14.226111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.227627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.296267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.298504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba639d-e740-462f-82f3-97e6bc725c4a service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.335119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1705] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:14.335119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061016-1ad1-4742-bbeb-9971387d6b0e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.335119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061016-1ad1-4742-bbeb-9971387d6b0e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.335119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be061016-1ad1-4742-bbeb-9971387d6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.372338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944f6fe-c4d4-490d-b57b-f2e2012d8235 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.387057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be2b5b0b240d4b5f82f5d52eb01dcb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b217ce25001e467f92a18edd1afa3cd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.388501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.388985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.389437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.390328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.394763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.400602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be061016-1ad1-4742-bbeb-9971387d6b0e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:14.402344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be061016-1ad1-4742-bbeb-9971387d6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.436846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:14.463860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 853/1706] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:14 2022] POST /identity/v3/projects => generated 355 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:14.465008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.571957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1707] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:14.581033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.583213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.583643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.617151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:14.618703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.635252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db308d-08ac-4682-b9ed-b942373fe436 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.641201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be2b5b0b240d4b5f82f5d52eb01dcb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b217ce25001e467f92a18edd1afa3cd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.641964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.642545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.643070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.643821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:14.644151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.646432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.665823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 854/1708] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:14.718932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.736743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d7632-667b-4b43-aa92-eeca3e0c7f1a tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.749158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be2b5b0b240d4b5f82f5d52eb01dcb1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b217ce25001e467f92a18edd1afa3cd0', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.750069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.750352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.750628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] PATH_INFO: `/v3/projects/f4e9890f5e634af4a2754f9ce5c0167d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.753220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f4e9890f5e634af4a2754f9ce5c0167d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.756371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.785517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1709] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:14.788257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.811083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299dc8e8-b5f0-481f-b0e9-2dbdd64e71d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.837820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.838515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.838792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.839064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.839797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:14.840152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.846447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.882429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1710] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:14.883787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.910811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab07244-0b98-4000-ac49-460985ab788d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.963567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e9890f5e634af4a2754f9ce5c0167d', 'request_id': 'req-bf789c18-9569-42f1-9b00-3f01f753b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:14.988819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e9890f5e634af4a2754f9ce5c0167d', 'request_id': 'req-bf789c18-9569-42f1-9b00-3f01f753b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:14.995487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:15.009548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e9890f5e634af4a2754f9ce5c0167d', 'request_id': 'req-bf789c18-9569-42f1-9b00-3f01f753b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:15.044670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] The token cache is being invalidate because project f4e9890f5e634af4a2754f9ce5c0167d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:15.046743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:15.054795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bf789c18-9569-42f1-9b00-3f01f753b879 tempest-VolumeQuotaClassesTest-1105470293 tempest-VolumeQuotaClassesTest-1105470293-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:15.057103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 855/1711] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:14 2022] DELETE /identity/v3/projects/f4e9890f5e634af4a2754f9ce5c0167d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:15.066069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473511-6d8c-45be-b8c9-46984f5b3fae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.067287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473511-6d8c-45be-b8c9-46984f5b3fae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.069805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06473511-6d8c-45be-b8c9-46984f5b3fae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.081969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 856/1712] 10.222.0.55 () {54 vars in 912 bytes} [Tue May 24 15:38:15 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:15.089845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.090131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.090450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.122455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.125205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.125750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.126909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.128119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.132740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.147223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:15.151262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:15.250680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1713] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:14 2022] POST /identity/v3/projects => generated 371 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:15.252584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.260355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:15.369520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 857/1714] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:15.389544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:15.419663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.421388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:15.435450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeef50e-cc95-48a5-b59b-cb1312534acc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.494928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802429-0137-4acc-a29e-8a30a4a44f4f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.509581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.510286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.510562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.510914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.511620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.519384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.576381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1715] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:15 2022] POST /identity/v3/users => generated 286 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:15.579483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.597978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.598917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.599207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.599544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.610951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec44796-f7d3-4afb-9abe-a5141a7fec83 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.627950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.628666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.628941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.629636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.629974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.631592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.635825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.637379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.655500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1716] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:15.659558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.662935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 858/1717] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:15.690717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d44bc-2cc5-415f-bb8f-500b07699593 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.703465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.704265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.704461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.704777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.705774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.709554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.725194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1718] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:15 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:15.726588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.749153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bec8a-8e3f-449c-b6f8-bbc92628bc60 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.760298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:15.761462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.761919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.762362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] PATH_INFO: `/v3/projects/065eb0050b8448889c43470fd59f0549/users/6ec41eb194664352860938f771d9e356/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.801160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065eb0050b8448889c43470fd59f0549, user_id=6ec41eb194664352860938f771d9e356, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:15.802363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:15.803929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409f299-7cdc-468f-a59f-d9d6e58951f3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:15.805404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:15.834306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ec3c-f80a-4818-86c2-17ba1064a350 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:15.845139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 859/1719] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:15 2022] PUT /identity/v3/projects/065eb0050b8448889c43470fd59f0549/users/6ec41eb194664352860938f771d9e356/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:15.848702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:15.849701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:15.850079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:15.956486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] MFA Rules not processed for user `6ec41eb194664352860938f771d9e356`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:15.961094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:16.005567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.006482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.007486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.007486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] PATH_INFO: `/v3/users/c63ef19910e1425faeec4314751a980f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.030470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c63ef19910e1425faeec4314751a980f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.035937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.124291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 860/1720] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:16.125227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.152061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff0ea7-9d73-45ef-b360-676d2e3f65d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:16.164701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.165388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.165667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.165944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.166643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.169985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.180756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:16.197549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c63ef19910e1425faeec4314751a980f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:16.219637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c63ef19910e1425faeec4314751a980f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:16.220588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c63ef19910e1425faeec4314751a980f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:16.221805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c63ef19910e1425faeec4314751a980f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:16.229825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 861/1721] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:16 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:16.233994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.239906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:16.254314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-69b34b68-30a7-4ece-b3cb-c99a16918e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c63ef19910e1425faeec4314751a980f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:16.288664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1722] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:15 2022] DELETE /identity/v3/users/c63ef19910e1425faeec4314751a980f => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:16.293273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.293561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.293840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.333287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:16.334534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:16.357784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:16.396292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c8030-3abf-41e6-adfc-636d1d7842df admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:16.513204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1723] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:16.521264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.522766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:16.552865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.553683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.554058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.554304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.556292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.558602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.621755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 862/1724] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:16 2022] POST /identity/v3/users => generated 292 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:16.625077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.639350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa813fc-fc00-4b00-a19a-732152f7e0bc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:16.648948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.649789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.650212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.650310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716bcaf-7eb2-4fa6-a625-bbfc67656266 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:16.650500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.651472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.653760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.669363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.670160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.672390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.672685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.685011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 863/1725] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:16.685997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.704821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.706961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.713255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d0b64c-ab4f-4f35-930e-f30cb17063d9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:16.724421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1726] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:16.728281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:16.729903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:16.730331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:16.730732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] PATH_INFO: `/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2/users/ef3ac2d7fb7d48b38f929ce87c988945/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:16.788590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cd213aa4284b048c58d3cef0bf5ec2, user_id=ef3ac2d7fb7d48b38f929ce87c988945, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:16.795086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:16.822876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 864/1727] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:16 2022] PUT /identity/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2/users/ef3ac2d7fb7d48b38f929ce87c988945/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:16.827032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:16.843476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42b0f7-b1f3-4f5b-a845-fe182b7ed19a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:17.047968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:17.048810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.049154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.049492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.050360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:17.053502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683a2eb-d341-459f-be4a-bdec488b9902 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:17.067204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1728] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:17.071262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:17.086475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6174703-0532-4e3e-bb87-9c6acd3a38c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:17.101274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:17.102392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.102987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.103677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] PATH_INFO: `/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2/users/ef3ac2d7fb7d48b38f929ce87c988945/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.105295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1cd213aa4284b048c58d3cef0bf5ec2, user_id=ef3ac2d7fb7d48b38f929ce87c988945, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:17.108986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf83491-7198-4641-bda6-f8b6d37cedfe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:17.136488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 865/1729] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:17 2022] PUT /identity/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2/users/ef3ac2d7fb7d48b38f929ce87c988945/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:17.147630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.147922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.148447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.211243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] MFA Rules not processed for user `ef3ac2d7fb7d48b38f929ce87c988945`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:17.212684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:17.309841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:17.390626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.391429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.391732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.411502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1730] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:17.423647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:17.437182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:17.456920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] MFA Rules not processed for user `6d07884e92bc4cfe95d2fd99c40f25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:17.463724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:17.554262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:17.623687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 866/1731] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:17.624759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:17.630299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:17.641225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ca12-49c1-4959-8c5c-3c5577d2b3a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:17.667867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:17.667867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.668318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.668582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] PATH_INFO: `/v3/projects/a22d23d88c81484da4ce2699565d6b85` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.672138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22d23d88c81484da4ce2699565d6b85)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:17.674612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:17.794204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4258c-3065-4b1e-a191-5be79a98b209 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:17.799709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:17.800343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.800631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.800912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.825498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:17.827178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a0dec-a052-4fcf-8de0-4f83c93ef78e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:17.841687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22d23d88c81484da4ce2699565d6b85', 'request_id': 'req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:17.848879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22d23d88c81484da4ce2699565d6b85', 'request_id': 'req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:17.860678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22d23d88c81484da4ce2699565d6b85', 'request_id': 'req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:17.865838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 867/1732] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:17.872328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:17.872643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:17.872927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:17.908511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] The token cache is being invalidate because project a22d23d88c81484da4ce2699565d6b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:17.910562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:17.916308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:17.919262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1733] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:17 2022] DELETE /identity/v3/projects/a22d23d88c81484da4ce2699565d6b85 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:17.928385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:17.931682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:17.947631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:17.957325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.097122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 868/1734] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:38:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:18.098467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:18.104193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89f8de-f03c-4d33-b3ea-23d59d9caf69 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:18.122221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.137388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:18.138545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:18.138835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:18.141167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] PATH_INFO: `/v3/users/be2b5b0b240d4b5f82f5d52eb01dcb1e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:18.145832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be2b5b0b240d4b5f82f5d52eb01dcb1e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:18.151125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:18.196338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:18.264667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be2b5b0b240d4b5f82f5d52eb01dcb1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:18.273824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002d1de-3b38-44ca-afb4-fe557cb624b9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:18.277456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be2b5b0b240d4b5f82f5d52eb01dcb1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:18.278395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be2b5b0b240d4b5f82f5d52eb01dcb1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:18.280440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be2b5b0b240d4b5f82f5d52eb01dcb1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:18.287439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:18.290307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:18.290732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:18.293230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:18.296804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.310892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be2b5b0b240d4b5f82f5d52eb01dcb1e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:18.335391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1735] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:17 2022] DELETE /identity/v3/users/be2b5b0b240d4b5f82f5d52eb01dcb1e => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:18.355944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:18.357762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.446020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:18.489783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:18.494591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:18.517268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 869/1736] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:18.540861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70308724-8254-4a10-803f-1fcad4d16e47 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:18.550301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a33be-3627-4099-a663-486114db5a8a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:18.551788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a33be-3627-4099-a663-486114db5a8a service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:18.552519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a33be-3627-4099-a663-486114db5a8a service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:18.553324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a33be-3627-4099-a663-486114db5a8a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:18.564415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-707a33be-3627-4099-a663-486114db5a8a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.661163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:18.662467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:18.700247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5da281-7cb3-48a3-bc42-51444097ca16 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:18.733043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a33be-3627-4099-a663-486114db5a8a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:18.737077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a33be-3627-4099-a663-486114db5a8a service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:18.779870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1737] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:38:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:38:18.890031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:18.891327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:18.895464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:18.898104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] PATH_INFO: `/v3/users/eff6a47a57734cc5828b6187d4cc908f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:18.905534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff6a47a57734cc5828b6187d4cc908f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:18.914569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:19.120645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff6a47a57734cc5828b6187d4cc908f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.146333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff6a47a57734cc5828b6187d4cc908f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.147618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff6a47a57734cc5828b6187d4cc908f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.175985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff6a47a57734cc5828b6187d4cc908f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.178732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff6a47a57734cc5828b6187d4cc908f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.252729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 870/1738] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:18 2022] DELETE /identity/v3/users/eff6a47a57734cc5828b6187d4cc908f => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:19.256923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:19.257887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:19.258368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:19.318181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:19.324071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:19.325924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:19.328050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:19.426428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:19.505831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd46bb-e191-4dfe-a01f-fb7c507ebfe3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:19.507388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1739] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:19.518908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:19.523196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:19.525771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:19.531021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:19.536221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:19.536568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] PATH_INFO: `/v3/projects/b217ce25001e467f92a18edd1afa3cd0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:19.542897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b217ce25001e467f92a18edd1afa3cd0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:19.553641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:19.562313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:19.679325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c21159-abb4-41fa-b8af-5c43c7951636 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:19.715709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b217ce25001e467f92a18edd1afa3cd0', 'request_id': 'req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.728396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b217ce25001e467f92a18edd1afa3cd0', 'request_id': 'req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.747669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b217ce25001e467f92a18edd1afa3cd0', 'request_id': 'req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.770449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] The token cache is being invalidate because project b217ce25001e467f92a18edd1afa3cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:19.772241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.773562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-828834d7-bc03-4ef8-9ad5-6a2cac2d7ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:19.793373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 871/1740] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:19 2022] DELETE /identity/v3/projects/b217ce25001e467f92a18edd1afa3cd0 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:19.798061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0322a-2dfb-4e97-ad7f-6c603322bef5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:19.798829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0322a-2dfb-4e97-ad7f-6c603322bef5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:19.800804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0322a-2dfb-4e97-ad7f-6c603322bef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:19.861397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:19.862293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:19.862575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:19.865779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:19.869053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:19.875508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-89d0322a-2dfb-4e97-ad7f-6c603322bef5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:19.879684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-89d0322a-2dfb-4e97-ad7f-6c603322bef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:20.063268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 872/1741] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:20.072440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.072969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.073262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.165573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:20.167477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.172260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:20.173958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.215975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 873/1742] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:20.219080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.231608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1743] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:20.235010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac0c59-0bd4-4953-98df-fbd87bb26031 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:20.243721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:20.244626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.244978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.245258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.246044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:20.246720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.249120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.271490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 874/1744] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:20.275273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.291017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabea08-6e03-4496-a9ed-95c6c6d2f64d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:20.318061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:20.318061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.318061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.318061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.318378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.318378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.345004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1745] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:20 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:20.349827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.378165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd98637-d0bb-4e8d-8c54-bfa1d09e1b36 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:20.627065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:20.627836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.628341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.628620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.629343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.631685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.697296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 875/1746] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:20 2022] POST /identity/v3/users => generated 292 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:20.702484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.737131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79aaf6-8e5f-42ce-b636-9810c2ef4261 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:20.752806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:20.756150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.756493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.756822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.760822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.763753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.820588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1747] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:20.822045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.827536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:20.840174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c890d0-1b1b-4da2-a613-1c7aefa10d55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:20.858903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:20.859737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:20.860023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:20.861455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:20.862195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:20.868879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:20.927611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 876/1748] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:20.929748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:20.959884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:20.984955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43d71c-4d39-438d-acc8-6e97b828ab3b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:21.009315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:21.010296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:21.010583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:21.010903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] PATH_INFO: `/v3/projects/193312023f99422fbec047d1e526a770/users/e70a1a4b8b884434b21a72b13b74f7ab/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:21.090961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=193312023f99422fbec047d1e526a770, user_id=e70a1a4b8b884434b21a72b13b74f7ab, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:21.101093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b834f5f3-e366-4476-8d1c-1f0004b0de13 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:21.151938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1749] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:20 2022] PUT /identity/v3/projects/193312023f99422fbec047d1e526a770/users/e70a1a4b8b884434b21a72b13b74f7ab/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:21.161036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:21.164617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:21.165617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:21.165617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:21.165945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:21.205578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9a0ea-1a04-4e65-ba99-fcf17b17165c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:21.230596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] MFA Rules not processed for user `e70a1a4b8b884434b21a72b13b74f7ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:21.231975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:21.412781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:21.413782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:21.414207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:21.414569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] PATH_INFO: `/v3/users/953fe7550e4349f6ae22896621590ceb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:21.416066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953fe7550e4349f6ae22896621590ceb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:21.418200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:21.511425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1750] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:21.516941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:21.550580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953fe7550e4349f6ae22896621590ceb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:21.553695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887fff7-4a7e-4dad-a7a5-d066809abcae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:21.574099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:21.578146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:21.578146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:21.578146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:21.582416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953fe7550e4349f6ae22896621590ceb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:21.583439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953fe7550e4349f6ae22896621590ceb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:21.591477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:21.602026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61762c61-90b9-464a-b24c-8471f2fd3b89 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:21.619188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953fe7550e4349f6ae22896621590ceb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:21.623153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953fe7550e4349f6ae22896621590ceb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:21.717389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1751] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:21 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:21.718981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae220-bb55-46b0-ae9e-d33c5a7aed13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:21.719313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae220-bb55-46b0-ae9e-d33c5a7aed13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:21.719713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeae220-bb55-46b0-ae9e-d33c5a7aed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:21.732267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 877/1752] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:21 2022] DELETE /identity/v3/users/953fe7550e4349f6ae22896621590ceb => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:21.737335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:21.741956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:21.784811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-abeae220-bb55-46b0-ae9e-d33c5a7aed13 None None] MFA Rules not processed for user `9bb20bf4c31d44ac9dd9ef0db9a2a63a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:21.787413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-abeae220-bb55-46b0-ae9e-d33c5a7aed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:22.019426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1169d9-f8f6-40a6-8a9a-6864e681ac56 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.030234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1753] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:22.030234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.030234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.030234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.034003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.035398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.035736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.036115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.037578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.043064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.082704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:22.084787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:22.089905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:22.106168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 878/1754] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:21 2022] POST /identity/v3/users => generated 298 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:22.113449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:22.123824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1755] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:22.149278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dce7b5-9100-4024-b41e-acf112cd9168 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.164775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.165696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.166097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.167987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.170852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.177658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.207201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 879/1756] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:22.212515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:22.236823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace9c49-4b9b-4357-b341-7b3386633bb4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.253841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.259004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.260676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.262177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] PATH_INFO: `/v3/projects/f66070861e5545c6acaac801cea23f63/users/e708f2cdab464d9b8ce9a61daf6c806e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.309150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66070861e5545c6acaac801cea23f63, user_id=e708f2cdab464d9b8ce9a61daf6c806e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.318080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.357319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1757] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:22 2022] PUT /identity/v3/projects/f66070861e5545c6acaac801cea23f63/users/e708f2cdab464d9b8ce9a61daf6c806e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:22.361950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:22.366874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:22.369445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:22.377684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddb5dd9-9a58-4b75-aab1-7084b76934f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.415860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:22.506108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66944bef-3e1f-421d-8f3c-72c2e763cbc5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.514043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.514886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.520875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.521474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.553837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.557016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.557990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.559869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.564344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.581727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.661660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 880/1758] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:22.668181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:22.709644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.711291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282fc933-beef-4c48-aad2-d66021b04edb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.731191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120cd99-fce8-496c-b4d5-27d311527405 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:22.741794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1759] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:22.761818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:22.762873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.763161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.763441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] PATH_INFO: `/v3/projects/f66070861e5545c6acaac801cea23f63/users/e708f2cdab464d9b8ce9a61daf6c806e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.776128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f66070861e5545c6acaac801cea23f63, user_id=e708f2cdab464d9b8ce9a61daf6c806e, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:22.792841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:22.847920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 881/1760] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:22 2022] PUT /identity/v3/projects/f66070861e5545c6acaac801cea23f63/users/e708f2cdab464d9b8ce9a61daf6c806e/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:22.860500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:22.860812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:22.861195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:22.933472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] MFA Rules not processed for user `e708f2cdab464d9b8ce9a61daf6c806e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:22.936349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.194651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1761] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:22 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:23.207239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:23.210996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.364352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:23.366156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.398177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51501b9-38a9-4e51-87b0-39984fcea47a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:23.408374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:23.409261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:23.409552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:23.409831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:23.423480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:23.424982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:23.436564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 882/1762] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:23.437718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:23.439789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.528128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:23.537427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8470c-aa78-4918-8312-cbe365a690c3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:23.544946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:23.546987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:23.547694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:23.548624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] PATH_INFO: `/v3/projects/2a6ea8ff8d114436bca2e300d9c01eea` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:23.550936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6ea8ff8d114436bca2e300d9c01eea)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:23.555749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:23.601469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb55c78-2e98-4d1d-8e20-dcc42b6e76fc service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:23.616432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:23.617687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:23.618181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:23.621690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] PATH_INFO: `/v3/users/b74d37f9e16a417baf03a8bbea2b5adf` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:23.626095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74d37f9e16a417baf03a8bbea2b5adf)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:23.632783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:23.668881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6ea8ff8d114436bca2e300d9c01eea', 'request_id': 'req-583f96d7-48a1-41c2-9e63-991b5d8b51b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.681567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:23.688860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6ea8ff8d114436bca2e300d9c01eea', 'request_id': 'req-583f96d7-48a1-41c2-9e63-991b5d8b51b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.714346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6ea8ff8d114436bca2e300d9c01eea', 'request_id': 'req-583f96d7-48a1-41c2-9e63-991b5d8b51b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.756583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] The token cache is being invalidate because project 2a6ea8ff8d114436bca2e300d9c01eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:23.757841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.759406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.772411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1763] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:23 2022] DELETE /identity/v3/projects/2a6ea8ff8d114436bca2e300d9c01eea => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:23.772411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:23.775936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.779311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74d37f9e16a417baf03a8bbea2b5adf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.797370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74d37f9e16a417baf03a8bbea2b5adf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.798396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74d37f9e16a417baf03a8bbea2b5adf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.818533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74d37f9e16a417baf03a8bbea2b5adf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.820698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de33cf77-fcc2-46ce-87bb-f65ba756cc11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74d37f9e16a417baf03a8bbea2b5adf'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:23.866750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 883/1764] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:23 2022] DELETE /identity/v3/users/b74d37f9e16a417baf03a8bbea2b5adf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:23.868253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614ad62-72f4-4d4c-9025-5e1e36083988 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:23.868579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614ad62-72f4-4d4c-9025-5e1e36083988 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:23.868803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614ad62-72f4-4d4c-9025-5e1e36083988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:23.917661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e614ad62-72f4-4d4c-9025-5e1e36083988 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:23.919468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e614ad62-72f4-4d4c-9025-5e1e36083988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:23.928078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583f96d7-48a1-41c2-9e63-991b5d8b51b8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.027720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.028433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.028711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.028983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.030835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.049179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 884/1765] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:24.050982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.051665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.052188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.078591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:24.080291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.094319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 885/1766] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:24.095743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.097627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.111562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd979f-aed4-434b-8d7b-5684e2885d18 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.118710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.119702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.120105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.120447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] PATH_INFO: `/v3/projects/ad0aa5dae88a4adfad862d64222f17ae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.121779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0aa5dae88a4adfad862d64222f17ae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.123768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.140398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.141917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4789d3-aab4-43b7-a65f-27032abbe235 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.146356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:24.157679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1767] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:24.160510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.160984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.161558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.207131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:24.208815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.240294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0aa5dae88a4adfad862d64222f17ae', 'request_id': 'req-4aab0856-1e63-4b15-9e60-9799b25fbe1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.246222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0aa5dae88a4adfad862d64222f17ae', 'request_id': 'req-4aab0856-1e63-4b15-9e60-9799b25fbe1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.259268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0aa5dae88a4adfad862d64222f17ae', 'request_id': 'req-4aab0856-1e63-4b15-9e60-9799b25fbe1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.291335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] The token cache is being invalidate because project ad0aa5dae88a4adfad862d64222f17ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:24.291668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.297189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.299442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 886/1768] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:24 2022] DELETE /identity/v3/projects/ad0aa5dae88a4adfad862d64222f17ae => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:24.302178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.310325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.352330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:24.397983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1769] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:24.399554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.402952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.431274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad0f6d-8c2e-475b-9c97-04984fa27d55 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.433281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.434271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.434545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.434813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] PATH_INFO: `/v3/users/db24427e62444d8e8380e9f779da4eeb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.436194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db24427e62444d8e8380e9f779da4eeb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.438471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.455820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab0856-1e63-4b15-9e60-9799b25fbe1f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.461428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.461428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.461428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.462015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.463926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.485211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.487922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.504444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 887/1770] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:24.506236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.507406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db24427e62444d8e8380e9f779da4eeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.517774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db24427e62444d8e8380e9f779da4eeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.519154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db24427e62444d8e8380e9f779da4eeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.519886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db24427e62444d8e8380e9f779da4eeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.542772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab5ec0-5f3e-4807-95b6-4ef62f5b7a46 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.566109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db24427e62444d8e8380e9f779da4eeb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:24.567665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.569179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.569502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.569780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.577323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:24.577736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.580359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.621227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1771] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:24 2022] DELETE /identity/v3/users/db24427e62444d8e8380e9f779da4eeb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:24.623188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 888/1772] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:24.634016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.637250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.844154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:24.847599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:24.900628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e79abe-4d7a-416a-b0ce-7342d9bfe3cd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.922501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.927970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.929570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.931969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] PATH_INFO: `/v3/users/e70a1a4b8b884434b21a72b13b74f7ab` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.936733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e70a1a4b8b884434b21a72b13b74f7ab)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.948377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb54ea-41e8-4ec6-8f12-9d41cbf5d1e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:24.949231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:24.964206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:24.964816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:24.967926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:24.968281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:24.969094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:24.981063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:25.042161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:25.095731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1773] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:24 2022] POST /identity/v3/projects => generated 365 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:25.096772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.118834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e70a1a4b8b884434b21a72b13b74f7ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.156658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e70a1a4b8b884434b21a72b13b74f7ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.157276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e70a1a4b8b884434b21a72b13b74f7ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.177578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e70a1a4b8b884434b21a72b13b74f7ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.180061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3248c132-f105-4d57-a4ee-c05e894bfd46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e70a1a4b8b884434b21a72b13b74f7ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.209037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 889/1774] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:24 2022] DELETE /identity/v3/users/e70a1a4b8b884434b21a72b13b74f7ab => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:25.211458 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:25.211808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:25.212106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:25.238770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:25.244114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.337516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957f24-9d18-4032-b636-d37f31da5183 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:25.409333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 890/1775] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:25.452562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.454140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.496362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:25.523852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:25.524959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:25.525269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:25.525676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:25.526831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:25.530223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:25.615043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2616a1-cf4c-48e3-886e-b511f6030217 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:25.615310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1776] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:25 2022] POST /identity/v3/users => generated 283 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:25.616518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.623488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:25.625791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:25.626666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:25.628202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:25.628750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.663121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:25.665041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:25.698386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee124e-c8d0-4343-806e-53eb534baf1f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:25.714302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 891/1777] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:25.715268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:25.715438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.719037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:25.719360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:25.721138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] PATH_INFO: `/v3/projects/f1b0c43108184906ba9f23c021f0345a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:25.721504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.725316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b0c43108184906ba9f23c021f0345a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:25.728811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:25.765891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e6f8f-4489-42a0-8c1b-a24bb0fa0159 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:25.779823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:25.780595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:25.780804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:25.781325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:25.782182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:25.786594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:25.818282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 892/1778] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:25.819244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.825159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.837905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b0c43108184906ba9f23c021f0345a', 'request_id': 'req-83dc1614-a26d-41d4-8406-d0d3c29fe432'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.849078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b0c43108184906ba9f23c021f0345a', 'request_id': 'req-83dc1614-a26d-41d4-8406-d0d3c29fe432'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.861851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b0c43108184906ba9f23c021f0345a', 'request_id': 'req-83dc1614-a26d-41d4-8406-d0d3c29fe432'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.864908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9633-cf72-43d4-9f97-0c89cb8ce638 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:25.880223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] The token cache is being invalidate because project f1b0c43108184906ba9f23c021f0345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:25.887565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.889540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:25.896209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1779] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:25 2022] DELETE /identity/v3/projects/f1b0c43108184906ba9f23c021f0345a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:25.896209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:25.896209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:25.939754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:26.051969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc1614-a26d-41d4-8406-d0d3c29fe432 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:26.075681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:26.077909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.078502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.079137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.080760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:26.084675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:26.085580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.085853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.086123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] PATH_INFO: `/v3/projects/1dced399c740478d9282fe6f2bf05a7d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.093233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dced399c740478d9282fe6f2bf05a7d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:26.094067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91f199-7e4d-4978-86a0-40631b4683f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:26.097515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:26.142784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1780] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:25 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:26.145129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7772-abac-41eb-9f05-b89e3b511408 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.145585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7772-abac-41eb-9f05-b89e3b511408 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.146147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe7772-abac-41eb-9f05-b89e3b511408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.197989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-71fe7772-abac-41eb-9f05-b89e3b511408 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:26.205514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71fe7772-abac-41eb-9f05-b89e3b511408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.241271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dced399c740478d9282fe6f2bf05a7d', 'request_id': 'req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.257427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dced399c740478d9282fe6f2bf05a7d', 'request_id': 'req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.276552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dced399c740478d9282fe6f2bf05a7d', 'request_id': 'req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.320163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] The token cache is being invalidate because project 1dced399c740478d9282fe6f2bf05a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:26.321874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.325048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.334066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 893/1781] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:25 2022] DELETE /identity/v3/projects/1dced399c740478d9282fe6f2bf05a7d => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:26.334066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:26.342170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.402978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1782] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:26.407045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.407512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.410278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.457027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:26.460433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.475412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:26.559471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1bc4b-a665-4d0f-9db3-1c6d08e2b456 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:26.576943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:26.578133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.578442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.578810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] PATH_INFO: `/v3/projects/01ad546228574f65b7370bbd8a8a2989/users/696f342f8c8842d580ded5a027c2bd1c/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.582437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1783] 10.222.0.59 () {58 vars in 1015 bytes} [Tue May 24 15:38:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:26.587734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:26.593387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.629862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ad546228574f65b7370bbd8a8a2989, user_id=696f342f8c8842d580ded5a027c2bd1c, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:26.633802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e95273-2e10-477b-b9c6-27a8d44f6291 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:26.664943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa90c7c-cc1c-415b-b799-505ea1c83e67 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:26.671920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 894/1784] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:26 2022] PUT /identity/v3/projects/01ad546228574f65b7370bbd8a8a2989/users/696f342f8c8842d580ded5a027c2bd1c/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:26.673538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.673828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.674103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.710685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:26.711939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.852124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:26.853031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.853309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.853582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] PATH_INFO: `/v3/users/570e486f8b01428296fcf8da7b993a80` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.854926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] RBAC: Authorizing `identity:delete_user(user_id=570e486f8b01428296fcf8da7b993a80)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:26.858367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:26.864547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:26.899696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 895/1785] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:26.901061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:26.902973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:26.916355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f6ac9-d61d-462a-84a2-6596c174c0d6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:26.921382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:26.922433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:26.922881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:26.923335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] PATH_INFO: `/v3/projects/193312023f99422fbec047d1e526a770` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:26.924792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=193312023f99422fbec047d1e526a770)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:26.927123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:26.933181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '570e486f8b01428296fcf8da7b993a80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.955920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '570e486f8b01428296fcf8da7b993a80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.958294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '570e486f8b01428296fcf8da7b993a80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.960450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '570e486f8b01428296fcf8da7b993a80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:26.983717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-091dfec1-72b8-46fe-89eb-c789e805b83b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '570e486f8b01428296fcf8da7b993a80'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.021497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '193312023f99422fbec047d1e526a770', 'request_id': 'req-775d7eeb-79b1-444c-9458-1cc9591d39a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.043139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1786] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:26 2022] DELETE /identity/v3/users/570e486f8b01428296fcf8da7b993a80 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:27.045852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.046314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.046740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.087612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '193312023f99422fbec047d1e526a770', 'request_id': 'req-775d7eeb-79b1-444c-9458-1cc9591d39a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.101015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '193312023f99422fbec047d1e526a770', 'request_id': 'req-775d7eeb-79b1-444c-9458-1cc9591d39a0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.120935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] The token cache is being invalidate because project 193312023f99422fbec047d1e526a770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:27.121261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.123084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.123681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] MFA Rules not processed for user `696f342f8c8842d580ded5a027c2bd1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:27.125212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 896/1787] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:26 2022] DELETE /identity/v3/projects/193312023f99422fbec047d1e526a770 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:27.125323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.132208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.134145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.231891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:27.257120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1788] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:27.258172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.259860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.283302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d7eeb-79b1-444c-9458-1cc9591d39a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:27.335456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f02b02-1e00-4a8b-ba80-74f0604c54ef None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:27.341355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:27.342269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.354161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.354568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] PATH_INFO: `/v3/users/e708f2cdab464d9b8ce9a61daf6c806e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.356019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e708f2cdab464d9b8ce9a61daf6c806e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:27.357987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:27.406234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:27.415315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:27.416006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.416324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.418220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.419105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:27.419519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:27.424171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:27.450435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 897/1789] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:27.450435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.450435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.502651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e708f2cdab464d9b8ce9a61daf6c806e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.518298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e708f2cdab464d9b8ce9a61daf6c806e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.519261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e708f2cdab464d9b8ce9a61daf6c806e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.520658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e708f2cdab464d9b8ce9a61daf6c806e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.540563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378314b-2f0c-4599-b2af-bd295695b98a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:27.551206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:27.551886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.552195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.552460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.557637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.597637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:27.599265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:27.623564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 898/1790] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:27.624591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.627739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.636070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e708f2cdab464d9b8ce9a61daf6c806e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:27.663967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8828d-aa9d-490e-b3ba-c2e2d635bc75 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:27.667530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1791] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:27 2022] DELETE /identity/v3/users/e708f2cdab464d9b8ce9a61daf6c806e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:27.668699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.680661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:27.795630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:27.797616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.798919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.799436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.803985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:27.811873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:27.837120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a7ffb-c8b4-44b0-905a-f7d021e17972 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:27.862618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:27.865869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:27.866747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:27.868373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] PATH_INFO: `/v3/projects/eaf20b463fd842f8819bf70f3280ec4c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:27.870269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 899/1792] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:27 2022] POST /identity/v3/projects => generated 373 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:27.871348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaf20b463fd842f8819bf70f3280ec4c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:27.874355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:27.881786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:27.914888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:28.003697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaf20b463fd842f8819bf70f3280ec4c', 'request_id': 'req-049cf8dd-fc67-42f3-ba6e-97651367a424'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.033256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaf20b463fd842f8819bf70f3280ec4c', 'request_id': 'req-049cf8dd-fc67-42f3-ba6e-97651367a424'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.049022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaf20b463fd842f8819bf70f3280ec4c', 'request_id': 'req-049cf8dd-fc67-42f3-ba6e-97651367a424'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.058063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2374c979-ef3d-48f0-a420-728cee9d6627 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.080442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] The token cache is being invalidate because project eaf20b463fd842f8819bf70f3280ec4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:28.080851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.081897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-049cf8dd-fc67-42f3-ba6e-97651367a424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.087409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1793] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:27 2022] DELETE /identity/v3/projects/eaf20b463fd842f8819bf70f3280ec4c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:28.095668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8091-9b5e-4b4c-a942-ead5db3d6fb8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.095668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8091-9b5e-4b4c-a942-ead5db3d6fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.095668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8091-9b5e-4b4c-a942-ead5db3d6fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.160063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbdb8091-9b5e-4b4c-a942-ead5db3d6fb8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:28.162360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbdb8091-9b5e-4b4c-a942-ead5db3d6fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.193997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:28.202881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.203848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.204184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.204449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.206402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.209906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.258292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 900/1794] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:27 2022] POST /identity/v3/users => generated 287 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:28.265862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.269411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.324018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1795] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:28.336221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10765-a119-4802-bbdc-deb88596e89f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.342052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.343644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.344579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.348005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.348940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.349887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.350833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.353090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.354872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.387198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 901/1796] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:28.396860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.398251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:28.399476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.419138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920320cf-ca2d-47d6-b915-c8ee72c46de9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.448453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1797] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:28.450622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.459478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.461154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.461154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.461154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.464715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.466375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.479808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75af03e-4c80-44d0-95ef-2c6eab1e987e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.489873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 902/1798] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:28.491069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.493495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.499434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.500225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.500597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.500869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.502536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:28.502851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.505612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.510103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeda262-ca58-4a15-8592-3538cd4d4459 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.516463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.517563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.517911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.518221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] PATH_INFO: `/v3/projects/f66070861e5545c6acaac801cea23f63` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.519629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f66070861e5545c6acaac801cea23f63)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.521847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.537656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1799] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:28.538836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.550039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef618afd-5efc-4f62-a69a-74f4bd1c5b34 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.564288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.567014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.567310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.569089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] PATH_INFO: `/v3/projects/1be5e4db69e84c1b921626e827e12f9c/users/f54db3056d994eb1991021ffe83179dc/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.620170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1be5e4db69e84c1b921626e827e12f9c, user_id=f54db3056d994eb1991021ffe83179dc, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.623182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.638309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f66070861e5545c6acaac801cea23f63', 'request_id': 'req-5649b941-dc2b-4971-94eb-717626da14bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.657872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f66070861e5545c6acaac801cea23f63', 'request_id': 'req-5649b941-dc2b-4971-94eb-717626da14bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.669263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1800] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:28 2022] PUT /identity/v3/projects/1be5e4db69e84c1b921626e827e12f9c/users/f54db3056d994eb1991021ffe83179dc/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:28.669554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.695884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f66070861e5545c6acaac801cea23f63', 'request_id': 'req-5649b941-dc2b-4971-94eb-717626da14bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.718136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] The token cache is being invalidate because project f66070861e5545c6acaac801cea23f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:28.719106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.720186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5649b941-dc2b-4971-94eb-717626da14bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:28.733412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 903/1801] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:28 2022] DELETE /identity/v3/projects/f66070861e5545c6acaac801cea23f63 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:28.735120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.735580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.736854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.773160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bff44-b22e-4afa-92d1-411787946d83 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.780995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:28.781745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.782068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.782326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.783501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:28.783983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:28.787835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307c361-6a03-4b12-99c5-e06d4b556675 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:28.797186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] MFA Rules not processed for user `f54db3056d994eb1991021ffe83179dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:28.800387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.827961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1802] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:28.829527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:28.829661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:28.830014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:28.855891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:28.858310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.890720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 904/1803] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:28.891947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:28.893463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:28.914459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8478c6-fff0-49dc-bd2e-6f50eca7ad6d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:28.989349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1804] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:28.990603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.005000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:29.017736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.018629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.019184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.019413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.020428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.022805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080bc09-35bc-4c19-969f-d87c0c0c88f6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.024007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:29.036892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d92e95-7ab0-4167-9472-1a833b3c4cf8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.047472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.048566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.048853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.049134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.049840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.051891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.063019 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 905/1805] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:29.064640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.065060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.065463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.088353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:29.089764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:29.101257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1806] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:28 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:29.101257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.105073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 906/1807] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:29.106009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.136451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46319a1-8853-4ef1-a787-619865dd10f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.204553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669f66f-3d62-4f79-960c-b745b4618097 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.211535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.212447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.213133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.213411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.214157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.215934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.219709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.220433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.220714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.220985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.221810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.223927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.232543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1808] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:29.233223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.245068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81d9c7-2c22-42cc-80f4-8236675c4b7a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.247105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 907/1809] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:29.247928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.249883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.250543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.250818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.251136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.251854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:29.252191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.253834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.267282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9c89e-53e3-4089-8d86-5c1f5652d7c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.268585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1810] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:29.269581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.281359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ada9a-ebd2-479c-9acc-0a330ec02017 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.282492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.283409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.283892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.284410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.285338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.287088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.288119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.288395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.288733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.289535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.289850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.291620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.299639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:29.307355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1811] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:29.307355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.309036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 908/1812] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:29.310032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.318779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072414c-2e30-49d3-b24c-cccab08073eb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.322342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401f4fc-1171-4828-831a-adefecc8a7a2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.324725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.325453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.325732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.325997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.326688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.328669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.332589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.333527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.333806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.334079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] PATH_INFO: `/v3/projects/82b6a2178169427eb69810afd9bb5813/users/b9a86c9cdae8419dbfee2b9b6cac4ba6/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.356000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b6a2178169427eb69810afd9bb5813, user_id=b9a86c9cdae8419dbfee2b9b6cac4ba6, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.358588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.363052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1813] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:29.363052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.382075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 909/1814] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:29 2022] PUT /identity/v3/projects/82b6a2178169427eb69810afd9bb5813/users/b9a86c9cdae8419dbfee2b9b6cac4ba6/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:29.383198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.489612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d1f20-50e9-4a62-8b4e-88759a29788c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.551031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:29.592336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.593663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.594122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.594496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] PATH_INFO: `/v3/projects/598fc585365a47df92a60234f2b737de/users/66a108575dbe45a7a2d6298ed587a6ef/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.615680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=598fc585365a47df92a60234f2b737de, user_id=66a108575dbe45a7a2d6298ed587a6ef, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.618526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.639491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1815] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:29 2022] PUT /identity/v3/projects/598fc585365a47df92a60234f2b737de/users/66a108575dbe45a7a2d6298ed587a6ef/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:29.648764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.721776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a9307-c97c-4ab9-9047-f6a647918cbf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.730081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.733738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.734160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.734648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.735559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.737720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8df9e1-ca6f-449b-b4c6-488f072a0d3b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.759507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1816] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:29.761718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f78a-6e29-485f-bb8d-de7e464df251 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.762536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f78a-6e29-485f-bb8d-de7e464df251 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.763603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8f78a-6e29-485f-bb8d-de7e464df251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.787997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d2d7c-5d97-4b8a-a258-6761102a1d8c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.794237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.794935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.795220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.795535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.797536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.803208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eba8f78a-6e29-485f-bb8d-de7e464df251 None None] MFA Rules not processed for user `66a108575dbe45a7a2d6298ed587a6ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:29.803627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.804686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eba8f78a-6e29-485f-bb8d-de7e464df251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:29.867742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 910/1817] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/users => generated 287 bytes in 485 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:29.868885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.889536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771130eb-ff48-44c0-af61-395e5b588fe2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:29.908872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:29.909964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.910519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.910897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] PATH_INFO: `/v3/projects/82b6a2178169427eb69810afd9bb5813/users/b9a86c9cdae8419dbfee2b9b6cac4ba6/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:29.914108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b6a2178169427eb69810afd9bb5813, user_id=b9a86c9cdae8419dbfee2b9b6cac4ba6, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:29.918152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:29.952089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 911/1818] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:29 2022] PUT /identity/v3/projects/82b6a2178169427eb69810afd9bb5813/users/b9a86c9cdae8419dbfee2b9b6cac4ba6/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:29.952089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:29.988672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1819] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:29.993473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:29.994266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:29.994996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.082007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] MFA Rules not processed for user `b9a86c9cdae8419dbfee2b9b6cac4ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:30.084107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:30.190581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:30.193495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4ebe3-1ba4-4916-bb55-52fce48b8ae8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.206771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.207762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.208130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.208403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.218102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.219810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.238588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1820] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:29 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:30.240425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.243665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 912/1821] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:29 2022] GET /identity/v3/roles => generated 1420 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:30.247247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.257995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e056a27-c809-4a84-8656-1ea4d9cd6bb3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.263326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.265180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.265543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.265976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.266137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb7e569-1e67-4f75-90ed-489058ea53b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.267200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.270769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.276719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.277841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.278114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.278481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.280248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.282756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.309997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 913/1822] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:30.311028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.314536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:30.331487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1823] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:30 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:30.335632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.375861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:30.440773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78d24-1c93-4a75-8ef7-b78e1f57a676 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.482082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7cf81-ada7-4e67-a988-4b25cd434d1a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.488248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.489423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.489844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.490543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] PATH_INFO: `/v3/projects/6023479b644c4cd59a8b60a0f35b44df/users/298e0e0af18c431eaf3f7699b020abdd/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.511319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6023479b644c4cd59a8b60a0f35b44df, user_id=298e0e0af18c431eaf3f7699b020abdd, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.526316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.535580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.536719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.538117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.542192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.546311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1824] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:30 2022] PUT /identity/v3/projects/6023479b644c4cd59a8b60a0f35b44df/users/298e0e0af18c431eaf3f7699b020abdd/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:30.547161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.645625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.647473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582afc09-4eb9-45e2-8ab1-c7daf5116858 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.671056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 914/1825] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:30.684475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.685923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.686277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.688408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2867ab7-c9fb-4436-b856-2f44da690ed7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.694387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.695771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.696269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.696918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.697796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.699972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.731990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1826] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:30 2022] POST /identity/v3/users => generated 291 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:30.739002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.751321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877c758f-2b44-4f15-8971-c76567dc1f91 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.752662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] MFA Rules not processed for user `298e0e0af18c431eaf3f7699b020abdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:30.754871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:30.766450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.767192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.767527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.767781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.768610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.770328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.808676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1827] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:30.814313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.842513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5cad2-8647-4b59-938e-737283c735c0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:30.853409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:30.854352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:30.854630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:30.854903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] PATH_INFO: `/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd/users/991862e0235a422284dd416d28f453a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:30.881574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 915/1828] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:30.881684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:30.891840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.897420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a48513bcbe4a4f9785b8a2635c5ffd, user_id=991862e0235a422284dd416d28f453a0, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:30.900743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:30.948398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1829] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:30 2022] PUT /identity/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd/users/991862e0235a422284dd416d28f453a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:30.949792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:30.975671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:31.012536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb9d129-c8f8-44f7-a42b-68bfe2ceb089 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.071556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542e16b-3be1-4fda-9cf6-67156dcd2ddb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.084858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.086012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.086012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.086298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.087007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.087528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.088247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.088531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.088675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.088858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.101587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1830] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:30 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:31.105246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.117092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770175-a651-4a9e-9a21-7a0c77bcee57 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.121991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.122895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.123358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.123431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] PATH_INFO: `/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd/users/991862e0235a422284dd416d28f453a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.125473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a48513bcbe4a4f9785b8a2635c5ffd, user_id=991862e0235a422284dd416d28f453a0, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.128149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40c74-89e3-4564-ad3c-67b4444b8d17 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.139433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1831] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:31 2022] PUT /identity/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd/users/991862e0235a422284dd416d28f453a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:31.165025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.165364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.165511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.165929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.167158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.176837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 916/1832] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:31.199415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.204921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] MFA Rules not processed for user `991862e0235a422284dd416d28f453a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:31.208861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.214929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff92897-8851-4213-9e1d-ae157dc28e4f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.280756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.281737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.282109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.282471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] PATH_INFO: `/v3/users/298e0e0af18c431eaf3f7699b020abdd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.283996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=298e0e0af18c431eaf3f7699b020abdd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.286101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.296141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1833] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:31.305779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.344872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:31.346637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298e0e0af18c431eaf3f7699b020abdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.355843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298e0e0af18c431eaf3f7699b020abdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.356601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298e0e0af18c431eaf3f7699b020abdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.364983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298e0e0af18c431eaf3f7699b020abdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.366324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-602ec98a-5ffa-4b46-bbf6-bc98023a47cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298e0e0af18c431eaf3f7699b020abdd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.366439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954ca9a-99ad-4052-b2a4-f51cd100870c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.370149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.370882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.371142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.371405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.374476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.387853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.389335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.389816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 917/1834] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:31 2022] DELETE /identity/v3/users/298e0e0af18c431eaf3f7699b020abdd => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:31.393908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:31.394033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.394237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.394507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.403677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1835] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:31.415834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:31.417089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.476566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.478186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.496409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e062e0-dfcd-4986-9baa-32b7ed07e8ad service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.505813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 918/1836] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:31.506924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.508619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.574708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.575216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f479a-6679-4d4d-98a1-a172a791fa89 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.575697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.576016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.576365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] PATH_INFO: `/v3/users/f54db3056d994eb1991021ffe83179dc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.577901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54db3056d994eb1991021ffe83179dc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.579592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.580272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.580558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.580834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.581239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.582941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.639149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54db3056d994eb1991021ffe83179dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.652090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54db3056d994eb1991021ffe83179dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.653063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54db3056d994eb1991021ffe83179dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.653854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54db3056d994eb1991021ffe83179dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.659559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.662393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.668387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fef47e4f-5ac3-4d78-9b8d-615ede10c264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54db3056d994eb1991021ffe83179dc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:31.675755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 919/1837] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:31.676849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.678379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.682750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1838] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:31 2022] DELETE /identity/v3/users/f54db3056d994eb1991021ffe83179dc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:31.690395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.693132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.693908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.709298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:31.716468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:31.721645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.760063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcf04e-8cbd-4246-8461-06b59bf56c56 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.794508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1839] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:31.808275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.809568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.840657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.841520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.841811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.842099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.844362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.847776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:31.863344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.865330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.879424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 920/1840] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:31.902706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.904304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.910721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8e423-8fdf-41b2-b5ba-f61cc1981a37 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.915857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.917018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.917784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.918304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.923092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4cd23-b0cb-4d5d-91ef-dfee4c1f2e1d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.929452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.931034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.935578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.936247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.936536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.936815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.943562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:31.945931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1841] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:31.947465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:31.962682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11f84a-fb0d-463f-87d5-fa6ceb11d5d7 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:31.968940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:31.972428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:31.973990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:31.978913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:31.981103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:31.981409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:31.995809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 921/1842] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:32.006190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:32.008165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:32.030328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1843] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:33.304045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:33.306335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:33.339324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f38ea-cde5-4671-bce3-85637e912220 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:33.349780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:33.352354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:33.352354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:33.354587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] PATH_INFO: `/v3/projects/1be5e4db69e84c1b921626e827e12f9c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:33.359321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1be5e4db69e84c1b921626e827e12f9c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:33.365176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:33.437057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:33.438466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:33.463406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1be5e4db69e84c1b921626e827e12f9c', 'request_id': 'req-d352c77f-418e-43b1-a058-b324c57b5df8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.474956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7d6be1-3230-4504-915e-686b8e331396 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:33.490713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1be5e4db69e84c1b921626e827e12f9c', 'request_id': 'req-d352c77f-418e-43b1-a058-b324c57b5df8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.501486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1be5e4db69e84c1b921626e827e12f9c', 'request_id': 'req-d352c77f-418e-43b1-a058-b324c57b5df8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.554830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] The token cache is being invalidate because project 1be5e4db69e84c1b921626e827e12f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:33.555125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.557059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.561967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 922/1844] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:33 2022] DELETE /identity/v3/projects/1be5e4db69e84c1b921626e827e12f9c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:33.569335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:33.571305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:33.668414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:33.683703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:33.685383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:33.685817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:33.686190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] PATH_INFO: `/v3/projects/6023479b644c4cd59a8b60a0f35b44df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:33.690684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6023479b644c4cd59a8b60a0f35b44df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:33.693381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:33.716758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352c77f-418e-43b1-a058-b324c57b5df8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:33.716758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:33.718425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:33.718749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:33.719105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] PATH_INFO: `/v3/users/b9a86c9cdae8419dbfee2b9b6cac4ba6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:33.720668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a86c9cdae8419dbfee2b9b6cac4ba6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:33.723414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:33.820332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a86c9cdae8419dbfee2b9b6cac4ba6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.835858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a86c9cdae8419dbfee2b9b6cac4ba6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.836824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a86c9cdae8419dbfee2b9b6cac4ba6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.866275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a86c9cdae8419dbfee2b9b6cac4ba6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.866786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6023479b644c4cd59a8b60a0f35b44df', 'request_id': 'req-ec9e610e-95c3-4092-a1bd-0cb06b110d17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.871129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ab5b6f6a-f0f3-4aed-8349-c4be3435d1bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a86c9cdae8419dbfee2b9b6cac4ba6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.886744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6023479b644c4cd59a8b60a0f35b44df', 'request_id': 'req-ec9e610e-95c3-4092-a1bd-0cb06b110d17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.911263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6023479b644c4cd59a8b60a0f35b44df', 'request_id': 'req-ec9e610e-95c3-4092-a1bd-0cb06b110d17'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.931022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 923/1845] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:33 2022] DELETE /identity/v3/users/b9a86c9cdae8419dbfee2b9b6cac4ba6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:33.937778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] The token cache is being invalidate because project 6023479b644c4cd59a8b60a0f35b44df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:33.938222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.939087 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec9e610e-95c3-4092-a1bd-0cb06b110d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:33.944578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1846] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:33 2022] DELETE /identity/v3/projects/6023479b644c4cd59a8b60a0f35b44df => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:33.946745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:33.948327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:33.948697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:33.978612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:33.980216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:34.073843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 924/1847] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:34.075648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.075871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.076197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.110636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:34.113960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:34.133549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1848] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:34.138353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.157005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322538d9-1716-4c46-8c27-1663e1d5aee8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.171367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.174834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.175285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.175569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.176569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:34.176818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.181273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.226742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 925/1849] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:34.231679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.245701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd856da-26b4-417e-8343-0184135d9086 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.261450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.262154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.264317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.264317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.267536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.272155 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25b58e-c565-49bb-b2e9-c0473857eb13 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.343333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1850] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:34 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:34.347328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.362672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2be5-3135-413d-882f-8f458dbee055 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.429574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39c2d1-ef5a-45e3-b302-bdbfa47f4684 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.430044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39c2d1-ef5a-45e3-b302-bdbfa47f4684 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.430456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39c2d1-ef5a-45e3-b302-bdbfa47f4684 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.433488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1851] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:38:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:34.520480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:34.529389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.530257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.530668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.554036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.554775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.555059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.555332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.557859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.561152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.581676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:34.596383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:34.607061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:34.623502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 926/1852] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:34 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:34.633825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.679771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77beb1c6-5791-4f58-b8cc-912f77f4e5b3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.697475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.698363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.698550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.698826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.699650 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.701743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.766351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 927/1853] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:34.771353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.786176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1854] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:38:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:34.793450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12464c-8acf-4206-9d3f-5d8202b9a9aa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.809656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.811925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.814697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.815986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.821313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.830045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c07ffc-a17d-43d3-b51e-48a243b84ef6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.863437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 928/1855] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:34.867164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:34.882804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d69c8-e7e4-4475-8e5c-a1898f66aacd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:34.900919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:34.903414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.903414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.903629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] PATH_INFO: `/v3/projects/80a3c5535f9040e58e055a7277514da1/users/0f0285314f124d228986e7f638833d04/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.922355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a3c5535f9040e58e055a7277514da1, user_id=0f0285314f124d228986e7f638833d04, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:34.926143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:34.950519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1856] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:34 2022] PUT /identity/v3/projects/80a3c5535f9040e58e055a7277514da1/users/0f0285314f124d228986e7f638833d04/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:34.951930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83251bf-6fe5-44ab-bd8d-6a42e4c91476 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:34.952267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83251bf-6fe5-44ab-bd8d-6a42e4c91476 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:34.952580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83251bf-6fe5-44ab-bd8d-6a42e4c91476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:34.986701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f83251bf-6fe5-44ab-bd8d-6a42e4c91476 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:34.990566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f83251bf-6fe5-44ab-bd8d-6a42e4c91476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:35.008426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:35.009963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:35.148101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 929/1857] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:35.150211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d485dc-6c13-426f-b1de-edb9c8087f2d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.150570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d485dc-6c13-426f-b1de-edb9c8087f2d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.150956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d485dc-6c13-426f-b1de-edb9c8087f2d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.154447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 930/1858] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:38:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:35.204794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.205334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.205799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.228636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c681527-31b3-47ad-a229-abd64488434b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:35.239075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:35.239757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.240058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.240342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.257854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:35.259405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:35.267098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:35.271545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:35.276581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 931/1859] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:38:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:35.289412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:35.293800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:35.298886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1860] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:35.458955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:35.477850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7e155-ba9f-4fc3-9877-10e555853ff8 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:35.481535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088baf86-fb0a-45ca-83c3-5ed52755c802 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:35.495461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:35.496723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.500022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.500929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:35.502174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.505780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.506496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.507958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.538753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:35.540624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:35.543842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:35.544069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd883f1-2ebc-4397-9de0-c007fe9e0c27 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:35.565975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1861] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:35.578692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 932/1862] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:36.079198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.080692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:36.107626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62e548-b420-4136-a498-d5dc49100ff3 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.126902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.128181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.128662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.129231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] PATH_INFO: `/v3/projects/82b6a2178169427eb69810afd9bb5813` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.130934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b6a2178169427eb69810afd9bb5813)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.133243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.219478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:36.229027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b6a2178169427eb69810afd9bb5813', 'request_id': 'req-1b258aba-4648-494d-8a31-6349f47fbe85'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:36.238575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b6a2178169427eb69810afd9bb5813', 'request_id': 'req-1b258aba-4648-494d-8a31-6349f47fbe85'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:36.262695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b6a2178169427eb69810afd9bb5813', 'request_id': 'req-1b258aba-4648-494d-8a31-6349f47fbe85'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:36.310719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] The token cache is being invalidate because project 82b6a2178169427eb69810afd9bb5813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:36.311081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:36.312510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1b258aba-4648-494d-8a31-6349f47fbe85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:36.317903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1863] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:36 2022] DELETE /identity/v3/projects/82b6a2178169427eb69810afd9bb5813 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:36.321235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.321565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.321940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.373563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:36.375386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:36.462565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:36.482565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 933/1864] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:36.483600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.484220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.484571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.517992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:36.520732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:36.539705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1865] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:36.543946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.559543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0687bf1-33d4-437e-b87c-9ee707acd432 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.567586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.568618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.569334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.569745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.570661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:36.571117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.573253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.592841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 934/1866] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:36.594718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.609891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075ee9-9919-45c7-b7dd-e83551ca72cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.645055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.646574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.647327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.648899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.651023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.657508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.718292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1867] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:36 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:36.719297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.733113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47027cc-c3da-4d8c-a143-31ebea91e14c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.803107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.805170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:36.816592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.817342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.817615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.817901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.818820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.821454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.852936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 935/1868] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:36 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:36.855803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.868415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab0df5-0427-4d89-8d4d-4308fc043460 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.874995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.875676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.875960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.876289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.877192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.880243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.893689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 936/1869] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:36.907178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.912656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d6a02-699f-4392-a114-6de19709c4a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.918638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.919873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.920387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.920858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] PATH_INFO: `/v3/users/66a108575dbe45a7a2d6298ed587a6ef` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.923001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a108575dbe45a7a2d6298ed587a6ef)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.926284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.929842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa43613-f9be-415b-9c4e-029c32e205ba admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.934825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.935519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.935760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.936080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:36.936776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:36.938385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.951777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 937/1870] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:36 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:36.956241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:36.972383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc0385-0e9e-42d4-9f9e-e04657a9d632 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:36.979593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:36.990074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:36.990512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:36.990835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] PATH_INFO: `/v3/projects/8f5149f6a2fb4a35a9193680831e5fbe/users/1f63cd0b315f4689b045e1d22381d78d/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.005465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a108575dbe45a7a2d6298ed587a6ef'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:37.026775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a108575dbe45a7a2d6298ed587a6ef'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:37.028532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f5149f6a2fb4a35a9193680831e5fbe, user_id=1f63cd0b315f4689b045e1d22381d78d, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.029500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a108575dbe45a7a2d6298ed587a6ef'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:37.030463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a108575dbe45a7a2d6298ed587a6ef'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:37.032406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63608aab-d927-4992-8c93-1c1d78de5373 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.063276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 938/1871] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:36 2022] PUT /identity/v3/projects/8f5149f6a2fb4a35a9193680831e5fbe/users/1f63cd0b315f4689b045e1d22381d78d/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:37.064658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-34093a5f-cb84-4ffc-848d-7599ef8a2ced admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a108575dbe45a7a2d6298ed587a6ef'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:37.065009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.065350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.065632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.089462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1872] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:36 2022] DELETE /identity/v3/users/66a108575dbe45a7a2d6298ed587a6ef => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:37.091081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.091620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.091897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.126721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] MFA Rules not processed for user `1f63cd0b315f4689b045e1d22381d78d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:37.127972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:37.137230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:37.147230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:37.178943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:37.254877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1873] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:37.308530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.310467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:37.325479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 939/1874] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:37.326628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.327976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:37.353006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4226576-ad7b-42d1-8a66-fb492041d6d4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.362695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.363858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.364441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.365214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.366402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.368672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.390664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:37.396272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad585d-73fe-46b9-8cbb-8eb1c65a7cbf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.401724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.404225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.404447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.404723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.418915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.418915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.424856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 940/1875] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:37 2022] POST /identity/v3/projects => generated 365 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:37.426153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.429499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1876] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:37.548321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05962fd-d40f-438a-aee6-cefdcf8fb6b3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.554763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.555583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.555866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.556167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.556967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.559033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.594061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 941/1877] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:37 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:37.596020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.608883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4df3d-8e9a-42dc-889d-e1f112e308cb service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.618460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.619242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.619589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.619971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.621185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.623683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.639145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1878] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:37.643549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.668238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdf36d-b201-43b7-acb9-40f9e79aab57 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.688401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.688401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.688401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.688776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] PATH_INFO: `/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809/users/7024083706fe4f29866434e14eeab511/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.737354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9c30e4b1a14232ae4e7cf17a8d0809, user_id=7024083706fe4f29866434e14eeab511, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.741057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.767090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 942/1879] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:37 2022] PUT /identity/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809/users/7024083706fe4f29866434e14eeab511/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:37.768346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.787799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f20478-ba51-47cc-a18f-3e57e162783c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.898217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.899163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.899609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.900020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.900936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.903444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1362187-06ca-4662-988b-54e08d4e5612 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.914578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1880] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:37.918725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:37.929780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6bb42-a39f-4d87-98fa-eee07a38aa39 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:37.934927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:37.935325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.935603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.935875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] PATH_INFO: `/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809/users/7024083706fe4f29866434e14eeab511/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:37.937890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9c30e4b1a14232ae4e7cf17a8d0809, user_id=7024083706fe4f29866434e14eeab511, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:37.940565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:37.954482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 943/1881] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:37 2022] PUT /identity/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809/users/7024083706fe4f29866434e14eeab511/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:37.965398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:37.965939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:37.966266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:38.008081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] MFA Rules not processed for user `7024083706fe4f29866434e14eeab511`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:38.011391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.070225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:38.157693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:38.158134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1882] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:38.159980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.326605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:38.328815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.360230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51828646-a22e-4284-a2ce-4590739a4e06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:38.364821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:38.365217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:38.365495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:38.365767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:38.382174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:38.388109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590a03d-6c8f-408d-8f30-ceccbd373703 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:38.397142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 944/1883] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:38.398773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:38.399259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:38.399725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:38.453416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] MFA Rules not processed for user `0f0285314f124d228986e7f638833d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:38.455064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.571461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d42e8-fecd-4576-9d14-23f2f62fbbba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:38.588105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:38.589652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:38.590094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:38.590952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:38.591458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] PATH_INFO: `/v3/projects/598fc585365a47df92a60234f2b737de` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:38.595328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=598fc585365a47df92a60234f2b737de)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:38.599796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:38.675103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 945/1884] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:38.677213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:38.680090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.712771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241aeb52-43f4-4e6f-8ccf-5d0f32b640dc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:38.747461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '598fc585365a47df92a60234f2b737de', 'request_id': 'req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.759074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '598fc585365a47df92a60234f2b737de', 'request_id': 'req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.774867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '598fc585365a47df92a60234f2b737de', 'request_id': 'req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.812343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] The token cache is being invalidate because project 598fc585365a47df92a60234f2b737de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:38.813691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.815324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.823667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1885] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:38 2022] DELETE /identity/v3/projects/598fc585365a47df92a60234f2b737de => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:38.830676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:38.832505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.847656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:38.848955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:38.849418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:38.849830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] PATH_INFO: `/v3/users/696f342f8c8842d580ded5a027c2bd1c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:38.851590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696f342f8c8842d580ded5a027c2bd1c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:38.854334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:38.916884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696f342f8c8842d580ded5a027c2bd1c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.927346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696f342f8c8842d580ded5a027c2bd1c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.928366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696f342f8c8842d580ded5a027c2bd1c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.938699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696f342f8c8842d580ded5a027c2bd1c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.939756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696f342f8c8842d580ded5a027c2bd1c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:38.965200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 946/1886] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:38 2022] DELETE /identity/v3/users/696f342f8c8842d580ded5a027c2bd1c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:38.966759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:38.968333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:38.971727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe33cbd-b5e0-421a-afaf-fab6c7bf1155 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:39.002444 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:39.101284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:39.102965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:39.103398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:39.104166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:39.106579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.133949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e141fb-af1e-4cbb-9ed1-e5b923e376b4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:39.142060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:39.143088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:39.143553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:39.144575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:39.148682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.176843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:39.188861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:39.191166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:39.215972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1887] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:39.217019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:39.224016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.300002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:39.302130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348dabba-101a-4ff9-9c83-950ae844ac97 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:39.321820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18aeb95-3f49-4053-a032-94933558022c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:39.323953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 947/1888] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:39.325485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:39.325991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:39.326262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:39.327401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:39.328669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:39.328964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:39.329240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] PATH_INFO: `/v3/users/991862e0235a422284dd416d28f453a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:39.330631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=991862e0235a422284dd416d28f453a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:39.333049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:39.345577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:39.347311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.360166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 948/1889] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:39.369609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:39.371510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.407148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '991862e0235a422284dd416d28f453a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:39.421257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '991862e0235a422284dd416d28f453a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:39.422377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '991862e0235a422284dd416d28f453a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:39.424907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '991862e0235a422284dd416d28f453a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:39.478748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '991862e0235a422284dd416d28f453a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:39.502496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:39.531617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1890] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:39 2022] DELETE /identity/v3/users/991862e0235a422284dd416d28f453a0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:39.585814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056fa5c-6ddf-47ef-aec6-be92677e80fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:39.744057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:39.749854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:39.750167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:39.750539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:39.755875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:39.851757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:39.854274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287f3ba-c58d-416f-b1c5-2f3f3ef7691a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:39.869373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 949/1891] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:40.470359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:40.471840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:40.486039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbfd97-8b81-4dbe-83ce-a7af400b9a7e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:40.492684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:40.494166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:40.494539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:40.494930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] PATH_INFO: `/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:40.496506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a48513bcbe4a4f9785b8a2635c5ffd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:40.499122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:40.584591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a48513bcbe4a4f9785b8a2635c5ffd', 'request_id': 'req-5a80c994-52e3-40f7-acb6-ddd827be41e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.591437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a48513bcbe4a4f9785b8a2635c5ffd', 'request_id': 'req-5a80c994-52e3-40f7-acb6-ddd827be41e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.604254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a48513bcbe4a4f9785b8a2635c5ffd', 'request_id': 'req-5a80c994-52e3-40f7-acb6-ddd827be41e7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.625284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] The token cache is being invalidate because project e9a48513bcbe4a4f9785b8a2635c5ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:40.629824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.633607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.637903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1892] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:40 2022] DELETE /identity/v3/projects/e9a48513bcbe4a4f9785b8a2635c5ffd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:40.643083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3241c-cd3a-4678-bfcb-15ccfbd738c6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:40.643687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3241c-cd3a-4678-bfcb-15ccfbd738c6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:40.644080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3241c-cd3a-4678-bfcb-15ccfbd738c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:40.662554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:40.665596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:40.684222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5fb3241c-cd3a-4678-bfcb-15ccfbd738c6 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:40.686548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5fb3241c-cd3a-4678-bfcb-15ccfbd738c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:40.725837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:40.819643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80c994-52e3-40f7-acb6-ddd827be41e7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:40.830043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:40.831420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:40.832049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:40.832335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] PATH_INFO: `/v3/projects/01ad546228574f65b7370bbd8a8a2989` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:40.837424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ad546228574f65b7370bbd8a8a2989)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:40.840388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:40.858830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 950/1893] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:40.866328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881f89-2abe-4558-98d4-3734ff62bee0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:40.866619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881f89-2abe-4558-98d4-3734ff62bee0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:40.866895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881f89-2abe-4558-98d4-3734ff62bee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:40.902352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-38881f89-2abe-4558-98d4-3734ff62bee0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:40.903762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-38881f89-2abe-4558-98d4-3734ff62bee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:40.944399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ad546228574f65b7370bbd8a8a2989', 'request_id': 'req-81bdc0b2-5665-4913-b560-9d6f035599cb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.950431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ad546228574f65b7370bbd8a8a2989', 'request_id': 'req-81bdc0b2-5665-4913-b560-9d6f035599cb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.963057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ad546228574f65b7370bbd8a8a2989', 'request_id': 'req-81bdc0b2-5665-4913-b560-9d6f035599cb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.979469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] The token cache is being invalidate because project 01ad546228574f65b7370bbd8a8a2989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:40.979803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.980520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-81bdc0b2-5665-4913-b560-9d6f035599cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:40.982381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1894] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:40 2022] DELETE /identity/v3/projects/01ad546228574f65b7370bbd8a8a2989 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:40.989478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:40.989838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:40.990204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.015972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:41.017404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:41.041139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:41.104492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 951/1895] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:41.105975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.106263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.106518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.125723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] MFA Rules not processed for user `0f0285314f124d228986e7f638833d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:41.125881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1896] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:41.126639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.127066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:41.146246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d18bc-e612-4677-a853-1ea4711a6965 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.154829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.155510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.155786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.156136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.156801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:41.157194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.159044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80164c-68bc-4f1c-bd0a-7bf5da803c01 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.180319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1897] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:41.182658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.183192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.183529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.234963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 952/1898] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:41.235712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:41.235899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.240973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:41.250781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8411f3-7696-48db-b810-45e7346a063d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.271325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1899] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:41.272055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.272756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.273038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.273315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.275173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.278092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.280138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.292904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b25e33-5fd1-434b-8961-db5a48ccc67b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.298216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.299276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.299276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.299464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.300608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:41.301198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.305206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.339260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1900] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:38:41.341017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.356918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 953/1901] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:41.361684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.384135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eefd74d-40ec-40be-a27e-6a07e6763c1b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.504061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:41.538242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.539405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.540394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.540734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.541861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.544152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf5719-40a0-45f6-b8db-5ef33cd4abb6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.544734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.555991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.556792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.557165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.557718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.558515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.562074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.588417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1902] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:41.593043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.599021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 954/1903] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/users => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:41.600726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.616906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f3ab93-825e-4232-b642-f5a5b528afd1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.681608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278c2dc-98a5-4084-9882-4da78f3a1a71 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.687589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.689230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.690359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.692633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.693744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.696042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.701015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.701788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.702133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.702475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.703636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.704895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:41.707824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.721269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 955/1904] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:41.725461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.729977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1905] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:41.731819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.737009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33e1a5-6943-4c3a-8915-c13595281113 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.744258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.744460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fbf867-009a-4348-a624-755f4eb89d76 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.745297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.745709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.746115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.747089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.749218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be9a97-3dc8-4a81-a9ad-0f8d87d830d9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.749953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.750535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.750858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.751426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.752287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.754787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.767774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 956/1906] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:41.769363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2b169-82a5-4d2e-a80e-ef4afa96ad6c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.769905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2b169-82a5-4d2e-a80e-ef4afa96ad6c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.770531 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2b169-82a5-4d2e-a80e-ef4afa96ad6c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.774363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1907] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:41.774536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 957/1908] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:38:41.774630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.775788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2b169-82a5-4d2e-a80e-ef4afa96ad6c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.786912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527836d-b1b8-469e-bb08-1717e977cfc9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.791608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2b169-82a5-4d2e-a80e-ef4afa96ad6c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.792619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.793638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.793931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.794202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] PATH_INFO: `/v3/projects/71fdda3a7c6e41ceb57a3cef30eec300/users/1d79d7ee1fe846ca8fa2bc82a88bb9ce/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.799147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.800169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.800702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.801351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.802265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.804984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.812854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fdda3a7c6e41ceb57a3cef30eec300, user_id=1d79d7ee1fe846ca8fa2bc82a88bb9ce, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.816068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46348423-076f-4b87-abb9-f7078b437935 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.821294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 958/1909] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:41.826759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:41.835140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1910] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:41 2022] PUT /identity/v3/projects/71fdda3a7c6e41ceb57a3cef30eec300/users/1d79d7ee1fe846ca8fa2bc82a88bb9ce/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:41.850153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.850153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.850435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.854639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d7ccc2-7319-41d2-8771-09f440e2125d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:41.895494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] MFA Rules not processed for user `1d79d7ee1fe846ca8fa2bc82a88bb9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:41.896940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:41.936048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:41.937383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.938275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.938275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] PATH_INFO: `/v3/projects/247c707f54a2424383c57b9f96298e2b/users/24865b55cd3a45d0b792e367b6a10376/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.955988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=247c707f54a2424383c57b9f96298e2b, user_id=24865b55cd3a45d0b792e367b6a10376, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:41.959356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2075-6f4b-46dc-9f16-a97355282dcf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:41.991427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 959/1911] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:41 2022] PUT /identity/v3/projects/247c707f54a2424383c57b9f96298e2b/users/24865b55cd3a45d0b792e367b6a10376/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:41.996254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1912] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:41.996383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd019cc-c8ac-4c12-992f-512dc8b1d344 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:41.996383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd019cc-c8ac-4c12-992f-512dc8b1d344 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:41.996383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd019cc-c8ac-4c12-992f-512dc8b1d344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:41.998500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.011591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85260a8e-7df9-48f6-9e2d-e91c1e7aecec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.018701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7bd019cc-c8ac-4c12-992f-512dc8b1d344 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:42.020326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7bd019cc-c8ac-4c12-992f-512dc8b1d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.049749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:42.089464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.090287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.090568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.091023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.091859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.094244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.126180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 960/1913] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:42.127906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.128485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.128986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.137999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1914] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:41 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:42.140959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.165707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] MFA Rules not processed for user `24865b55cd3a45d0b792e367b6a10376`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:42.166929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.198380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:42.240346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66492320-faf5-4397-8ead-3c05417ded81 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.245279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.245979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.246223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.246542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.247264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.249289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.252152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 961/1915] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:42.252989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.254781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.272863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1916] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:42 2022] POST /identity/v3/users => generated 306 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:42.273676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.284441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3d21d-6a55-41d5-bdcb-caadd9bf3ae4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.289438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.290102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.290558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.290631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.291327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.293288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.325560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1917] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:42 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:42.326199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.349574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a763d0-e831-4a1f-9c22-d47238e4cda1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.402892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:42.431070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48147a8e-6885-4901-881b-b98c85c8cd64 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.436417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.437197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.437545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.437985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.438694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.440788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.443903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.444554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.444833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.445104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.455239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1918] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:42.456317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.458641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.460508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.482043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75492a-e425-4016-af06-7ffa62cb187e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.496440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.497436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.497830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.498174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.498388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 962/1919] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:42.499405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.501003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.502345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.530971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073ab47-05b2-45da-a92f-6be20c672307 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.534651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1920] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:42 2022] POST /identity/v3/users => generated 299 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:42.539478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.542592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.543590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.543776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.544083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] PATH_INFO: `/v3/projects/ca2378176a79476f81b2996c722d57db/users/19c0bf99faaf49a7b270e2e8696b2049/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.563167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b31410d-d898-4cd2-8caa-d428e0517021 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.583992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2378176a79476f81b2996c722d57db, user_id=19c0bf99faaf49a7b270e2e8696b2049, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.585022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.585716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.586047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.586317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.587053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.588858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.588985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.603929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1921] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:42.605493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.607701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.617958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 963/1922] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:42 2022] PUT /identity/v3/projects/ca2378176a79476f81b2996c722d57db/users/19c0bf99faaf49a7b270e2e8696b2049/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:42.617958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.646877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043d4e2-6833-4b43-bd1c-d9dd0004de85 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.714011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db62af9-f55a-437f-b42b-c072e25a2e1c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.719802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.721247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.721604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.723497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] PATH_INFO: `/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506/users/87deed43026040cb8c5035fd7a794bd2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.748455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cca00a0f7b04d6f9d79013f1b8fc506, user_id=87deed43026040cb8c5035fd7a794bd2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.751250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.757492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.759630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.759891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.760266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] PATH_INFO: `/v3/users/1f63cd0b315f4689b045e1d22381d78d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.761584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f63cd0b315f4689b045e1d22381d78d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.764064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.770373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:42.778195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 964/1923] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:42 2022] PUT /identity/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506/users/87deed43026040cb8c5035fd7a794bd2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:42.778195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.845791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f63cd0b315f4689b045e1d22381d78d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:42.857157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f63cd0b315f4689b045e1d22381d78d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:42.857821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f63cd0b315f4689b045e1d22381d78d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:42.858529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f63cd0b315f4689b045e1d22381d78d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:42.867123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cfa46f-ca5c-467a-b626-a1e09982e0c2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.874448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f63cd0b315f4689b045e1d22381d78d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:42.876904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.877309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.877632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.877991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.878726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.880812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9c8c-638a-417a-b59c-089da3d5484e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.888720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1924] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:42 2022] DELETE /identity/v3/users/1f63cd0b315f4689b045e1d22381d78d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:42.892510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 965/1925] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:42.893701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.893989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.894265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.895341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.896754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.913229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:42.914599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.934528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:42.968914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2674c79-4d88-418a-a56a-206732c7825f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.974793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:42.975484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:42.975758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:42.976046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:42.976782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:42.978388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:42.980360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 966/1926] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:42.981181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:42.982581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:42.994807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9916ba9-4b5e-40c1-be93-0d3cd5b9a0d7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:42.999922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.001006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.001651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1927] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:43.001832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.002359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] PATH_INFO: `/v3/projects/ca2378176a79476f81b2996c722d57db/users/19c0bf99faaf49a7b270e2e8696b2049/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.002561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.004340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.004897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2378176a79476f81b2996c722d57db, user_id=19c0bf99faaf49a7b270e2e8696b2049, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.007315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.029805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 967/1928] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:42 2022] PUT /identity/v3/projects/ca2378176a79476f81b2996c722d57db/users/19c0bf99faaf49a7b270e2e8696b2049/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:43.029805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.079808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1e5d6-ca00-4cd5-85a1-2f5e88c04a45 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.098694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d1234-ac1d-431c-bbe1-9a55ad481159 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.103291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.104209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.104486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.104752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] PATH_INFO: `/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506/users/87deed43026040cb8c5035fd7a794bd2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.108057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:43.108944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cca00a0f7b04d6f9d79013f1b8fc506, user_id=87deed43026040cb8c5035fd7a794bd2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.112557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572ccdd-e2bd-479c-900f-d06703e47b42 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.123983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 968/1929] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:43 2022] PUT /identity/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506/users/87deed43026040cb8c5035fd7a794bd2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:43.128197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.128197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.128197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.131437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.132332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.132947 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.133224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.142667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.144133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d544389-d52e-4ad8-880e-1bd2434f1e2b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.154374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] MFA Rules not processed for user `19c0bf99faaf49a7b270e2e8696b2049`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:43.154539 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1930] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:43.155581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.155789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.156105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.156384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.183210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] MFA Rules not processed for user `87deed43026040cb8c5035fd7a794bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:43.184485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.232496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 969/1931] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:43.244307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.246547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.273787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1932] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:43.285376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.286789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.291775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:43.323405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fe13b-145d-4778-b00b-5ecf82bb4833 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.328180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.328842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.329281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.329561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.339659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.341180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.351579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 970/1933] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:43.354302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088319e2-cfa3-444f-9107-7fa40cb08390 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.359773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.360609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.360936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.361212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.373571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.375323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.389538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1934] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:43.417387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.433528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.443713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16dc5d-5a22-42c0-9a85-97fd9f8ef289 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.448234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.448895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.449178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.449452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.451969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.509645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffaed1-663b-4682-b1d0-95f8053b27a9 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.515431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.516440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.516966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.517312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] PATH_INFO: `/v3/users/24865b55cd3a45d0b792e367b6a10376` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.518977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24865b55cd3a45d0b792e367b6a10376)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.524992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.536961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.538426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.548503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1935] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:43.549356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.550869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.568309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c673a-6ec4-4c10-ba8c-d58e2b824de8 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.573983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.574877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.575415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.575681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] PATH_INFO: `/v3/projects/8f5149f6a2fb4a35a9193680831e5fbe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.577142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f5149f6a2fb4a35a9193680831e5fbe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.579125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.580889 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:43.592739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24865b55cd3a45d0b792e367b6a10376'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.601119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24865b55cd3a45d0b792e367b6a10376'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.601791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24865b55cd3a45d0b792e367b6a10376'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.610495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24865b55cd3a45d0b792e367b6a10376'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.611282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-29371ad9-40e4-42a7-9448-a8e751570970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24865b55cd3a45d0b792e367b6a10376'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.632701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 971/1936] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:43 2022] DELETE /identity/v3/users/24865b55cd3a45d0b792e367b6a10376 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:43.637375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.637814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.638093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.638582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f5149f6a2fb4a35a9193680831e5fbe', 'request_id': 'req-e2ec1706-bf5d-4091-9a45-60cdca3718d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.644891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f5149f6a2fb4a35a9193680831e5fbe', 'request_id': 'req-e2ec1706-bf5d-4091-9a45-60cdca3718d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.654810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f5149f6a2fb4a35a9193680831e5fbe', 'request_id': 'req-e2ec1706-bf5d-4091-9a45-60cdca3718d4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.656065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:43.657494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.669291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] The token cache is being invalidate because project 8f5149f6a2fb4a35a9193680831e5fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:43.669584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.670276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.672085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1937] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:43 2022] DELETE /identity/v3/projects/8f5149f6a2fb4a35a9193680831e5fbe => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:43.676183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.677607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.711268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:43.728232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 972/1938] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:43.738623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:43.740055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:43.754922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1706-bf5d-4091-9a45-60cdca3718d4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.760055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.760894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.761171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.761497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] PATH_INFO: `/v3/users/7024083706fe4f29866434e14eeab511` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.762684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7024083706fe4f29866434e14eeab511)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.764681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.814557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b989651-21aa-498e-92ad-0766e0037a38 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:43.821779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:43.824942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7024083706fe4f29866434e14eeab511'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.828793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:43.829079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:43.829356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:43.847810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7024083706fe4f29866434e14eeab511'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.847810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7024083706fe4f29866434e14eeab511'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.847810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7024083706fe4f29866434e14eeab511'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.852610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:43.857544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:43.870387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7024083706fe4f29866434e14eeab511'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:43.882059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 973/1939] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:43.890299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1940] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:43 2022] DELETE /identity/v3/users/7024083706fe4f29866434e14eeab511 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:44.177945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.179532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.202555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ec09e-5c58-47ec-b4db-74c745423a96 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.282131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.283069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.283297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.283620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] PATH_INFO: `/v3/projects/247c707f54a2424383c57b9f96298e2b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.284896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=247c707f54a2424383c57b9f96298e2b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.287101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.344561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247c707f54a2424383c57b9f96298e2b', 'request_id': 'req-3f80f992-b39d-4bdb-8c57-421b023961a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.351045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247c707f54a2424383c57b9f96298e2b', 'request_id': 'req-3f80f992-b39d-4bdb-8c57-421b023961a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.362842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247c707f54a2424383c57b9f96298e2b', 'request_id': 'req-3f80f992-b39d-4bdb-8c57-421b023961a2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.368646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.370923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.376023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] The token cache is being invalidate because project 247c707f54a2424383c57b9f96298e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:44.376399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.377450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.379605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 974/1941] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:44 2022] DELETE /identity/v3/projects/247c707f54a2424383c57b9f96298e2b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:44.383554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.385203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.397633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8aba9-bf9a-4308-9e5e-791fe6b24120 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.404403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:44.417643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:44.453783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80f992-b39d-4bdb-8c57-421b023961a2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.458627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.459513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.459795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.460100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] PATH_INFO: `/v3/users/87deed43026040cb8c5035fd7a794bd2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.461286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87deed43026040cb8c5035fd7a794bd2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.461613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.462489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.462766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.463037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] PATH_INFO: `/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.463252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.464247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9c30e4b1a14232ae4e7cf17a8d0809)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.466156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.512528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87deed43026040cb8c5035fd7a794bd2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.513523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9c30e4b1a14232ae4e7cf17a8d0809', 'request_id': 'req-e836b668-6011-4f78-af8c-ed0fd176256e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.520289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9c30e4b1a14232ae4e7cf17a8d0809', 'request_id': 'req-e836b668-6011-4f78-af8c-ed0fd176256e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.522273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87deed43026040cb8c5035fd7a794bd2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.522984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87deed43026040cb8c5035fd7a794bd2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.529130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9c30e4b1a14232ae4e7cf17a8d0809', 'request_id': 'req-e836b668-6011-4f78-af8c-ed0fd176256e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.532422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87deed43026040cb8c5035fd7a794bd2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.533229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a59d8000-9454-404b-9da9-d6d6d43137a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87deed43026040cb8c5035fd7a794bd2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.542932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] The token cache is being invalidate because project 3e9c30e4b1a14232ae4e7cf17a8d0809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:44.543351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.544009 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e836b668-6011-4f78-af8c-ed0fd176256e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.548903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1942] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:44 2022] DELETE /identity/v3/projects/3e9c30e4b1a14232ae4e7cf17a8d0809 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:44.550236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.550521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.550798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.556648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 975/1943] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:44 2022] DELETE /identity/v3/users/87deed43026040cb8c5035fd7a794bd2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:44.569564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:44.570776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.641289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1944] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:44.649170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.649461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.650124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.677513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:44.678883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.703438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 976/1945] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:44.709281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.719718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e521c4-9dd6-4c27-bd03-abeb249a56b1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.724609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.725307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.725589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.725856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.726584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:44.726886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.728526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.744794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1946] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:44.746008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.755438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22106f-083c-4228-b7c3-87f0150acb74 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.760338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.761034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.761711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.761711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.762297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.764426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.795915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 977/1947] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:44 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:44.800004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.810814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05058e0-ece8-448f-8f53-5091b3d65944 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.835699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:44.870897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.871226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.871570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.871912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.872701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.874727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.894634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1948] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:44 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:44.899037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.911730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21232b-a6e1-4484-b5f2-14c1576dbb7b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.911893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.913725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:44.917441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.918375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.918750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.919116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.919946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.921949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.927924 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace9cea8-8c42-4bd5-a32d-77420fa07bc9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.933039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.933861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 978/1949] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:44.933974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.934191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.934466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] PATH_INFO: `/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.935688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cca00a0f7b04d6f9d79013f1b8fc506)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.937700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.946431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.957546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b664b5c-aefc-4920-880b-a21b9743301e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:44.962588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:44.963254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:44.963530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:44.963798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:44.964692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:44.966285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:44.977721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 979/1950] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:44.982174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:44.993255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cca00a0f7b04d6f9d79013f1b8fc506', 'request_id': 'req-bf136be1-272a-41d6-adef-d477c3465403'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:44.999074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cca00a0f7b04d6f9d79013f1b8fc506', 'request_id': 'req-bf136be1-272a-41d6-adef-d477c3465403'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:45.004682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86316e5d-073e-41ed-858f-0a1c661403c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.007747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cca00a0f7b04d6f9d79013f1b8fc506', 'request_id': 'req-bf136be1-272a-41d6-adef-d477c3465403'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:45.025039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] The token cache is being invalidate because project 8cca00a0f7b04d6f9d79013f1b8fc506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:45.025538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:45.026540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bf136be1-272a-41d6-adef-d477c3465403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:45.033391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1951] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:44 2022] DELETE /identity/v3/projects/8cca00a0f7b04d6f9d79013f1b8fc506 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:45.037080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64260f-3cef-447c-8f75-2370c4f9348e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.037080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64260f-3cef-447c-8f75-2370c4f9348e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.037080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb64260f-3cef-447c-8f75-2370c4f9348e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.058807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb64260f-3cef-447c-8f75-2370c4f9348e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:45.059705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:45.060873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb64260f-3cef-447c-8f75-2370c4f9348e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.068272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.069747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.070490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.070830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] PATH_INFO: `/v3/projects/7d21540f6ac14ac5a591f0e16a8c6c1c/users/afeb858ab0db49c08bb1f7cfb4ce8942/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.094739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d21540f6ac14ac5a591f0e16a8c6c1c, user_id=afeb858ab0db49c08bb1f7cfb4ce8942, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.097740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688eb71-0157-48f7-976a-1f290c2075ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.112458 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 980/1952] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:44 2022] PUT /identity/v3/projects/7d21540f6ac14ac5a591f0e16a8c6c1c/users/afeb858ab0db49c08bb1f7cfb4ce8942/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:45.123630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.123917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.124227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.136925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1953] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:45.139134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.139134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.139416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.151669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] MFA Rules not processed for user `afeb858ab0db49c08bb1f7cfb4ce8942`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:45.154038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.167553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:45.168870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.216598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:45.236084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 981/1954] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:45.241479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.243421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.251898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1955] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:45.256058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.258519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee314207-bc6a-47dc-84ca-74285b2742cd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.263540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.264256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.264541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.264813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.265517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.267696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.270093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da802b-940b-4ead-9fd6-68a83e3101c4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.278106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.279606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.280131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.280655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.281457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:45.281829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.283607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.300124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1956] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:38:45.303305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.311529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 982/1957] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:45.312436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.314842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb7689-bdb7-4638-b650-55f480f303ef admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.399829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8420de2-a4fb-4227-9977-85f337388d87 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.406343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.407126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.407727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.408818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.409034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.412498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.448599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 983/1958] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:45.451504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.452380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.452488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.452752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.453072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.453785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.455916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.475012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c03ed-8427-4653-95eb-ad9265595188 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.479814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.480522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.480803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.481082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.481992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.483673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.500118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 984/1959] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:45.501887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.508150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1960] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/projects => generated 367 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:45.512557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.520017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f4f98-951f-4a73-a3da-d0cd27e80983 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.556001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:45.591294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c80fadb-c9b3-42f0-8716-04e1562dfc72 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.596400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.597075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.597584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.597866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.598559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.600733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.600878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.601883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.602226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.602564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] PATH_INFO: `/v3/projects/ff9c21c940b943458ee6feafbfbcb273/users/a934a9ed1bcc485e80e1f94ae4867480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.620494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9c21c940b943458ee6feafbfbcb273, user_id=a934a9ed1bcc485e80e1f94ae4867480, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.623250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.628067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1961] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/users => generated 284 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:45.632265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.638078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 985/1962] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:45 2022] PUT /identity/v3/projects/ff9c21c940b943458ee6feafbfbcb273/users/a934a9ed1bcc485e80e1f94ae4867480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:45.641656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.643647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682aed8-8a06-49c7-a379-acaa2d4159d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.697981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:45.708809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d90080-2e27-4ca1-b855-bef37125fbef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.716403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.717106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.717459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.717738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.718464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.720093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.721911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.722924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.723208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.723485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.724360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.728338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.733440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1963] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:45.734543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.738460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 986/1964] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:45.742489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.745702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08dd882-11ab-4f50-984d-bee389ece55b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.750939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.751620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.751906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.752211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.753108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.753620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0930f0c-cdfc-481e-a414-a53a1eb778a4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.755112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.758659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.759555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.759834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.760144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] PATH_INFO: `/v3/projects/ff9c21c940b943458ee6feafbfbcb273/users/a934a9ed1bcc485e80e1f94ae4867480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.762125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9c21c940b943458ee6feafbfbcb273, user_id=a934a9ed1bcc485e80e1f94ae4867480, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.765499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bfbcd-7c0d-43e7-b815-a5d5179af49c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.766710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1965] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:45.770681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.786370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 987/1966] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:45 2022] PUT /identity/v3/projects/ff9c21c940b943458ee6feafbfbcb273/users/a934a9ed1bcc485e80e1f94ae4867480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:45.786778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd7ecb-321e-484d-90db-17900cd48422 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.793134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.793482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.793808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.828481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] MFA Rules not processed for user `a934a9ed1bcc485e80e1f94ae4867480`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:45.829643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.839396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:45.840342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.840621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.840893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] PATH_INFO: `/v3/projects/6f55e272091b4ed8ab985f82e710aae8/users/6f967a03794048e2b0023c4e42efbe18/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.855767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f55e272091b4ed8ab985f82e710aae8, user_id=6f967a03794048e2b0023c4e42efbe18, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:45.858474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9f504-6c51-4234-921d-955be920f0fc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:45.871484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1967] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:45 2022] PUT /identity/v3/projects/6f55e272091b4ed8ab985f82e710aae8/users/6f967a03794048e2b0023c4e42efbe18/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:45.882911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:45.883198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:45.883473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:45.899864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 988/1968] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:45.910441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] MFA Rules not processed for user `6f967a03794048e2b0023c4e42efbe18`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:45.911655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.912053 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:45.913485 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:45.932991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:45.975123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265205dd-fad2-441a-b71a-b24e4341b6b7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:45.979370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1969] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:45.983853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.024796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.025456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.025734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.026058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.027497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:46.049445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bffd6b-3e69-4e43-8a2a-23a52a9a7e22 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.054394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.055062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.055343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.055616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.056344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.058307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.084549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.084822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1970] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:45 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:46.086006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.088738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.095553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 989/1971] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:38:46.151192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f585-c07d-4327-963d-677ace4fb087 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.156072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.156735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.157011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.157280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.157988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.160067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.179791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1972] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:46 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:46.184049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.195418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a67e2-0c68-479a-8e76-1f3aaeee8284 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.200095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.200757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.201032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.201300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.202011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.203599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.213820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 990/1973] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:46.218441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.230293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c06887-f217-4221-9d8b-ac0251bdfc14 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.235097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.235991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.236312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.236585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] PATH_INFO: `/v3/projects/7eccc882f3274a61a7f437b5e9283213/users/b3ef6f26ce2c43aaabe53552ef61f6ec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.252709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eccc882f3274a61a7f437b5e9283213, user_id=b3ef6f26ce2c43aaabe53552ef61f6ec, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.255421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.273112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1974] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:46 2022] PUT /identity/v3/projects/7eccc882f3274a61a7f437b5e9283213/users/b3ef6f26ce2c43aaabe53552ef61f6ec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:46.277704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.289588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1feafaf-f20c-4695-a497-7b2ed66b8e6b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.355947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.356947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.357291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.357569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.358479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.360506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b9bc9-c193-455f-8cb9-67c3ae6d6fb3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.371476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 991/1975] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:46.375392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.386161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe29b53-0927-44b2-90e7-40cd8a70f851 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.392015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.392977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.393257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.393533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] PATH_INFO: `/v3/projects/7eccc882f3274a61a7f437b5e9283213/users/b3ef6f26ce2c43aaabe53552ef61f6ec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.395700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eccc882f3274a61a7f437b5e9283213, user_id=b3ef6f26ce2c43aaabe53552ef61f6ec, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.398437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.409788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1976] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:46 2022] PUT /identity/v3/projects/7eccc882f3274a61a7f437b5e9283213/users/b3ef6f26ce2c43aaabe53552ef61f6ec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:46.421367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.421658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.421936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.441902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:46.449666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] MFA Rules not processed for user `b3ef6f26ce2c43aaabe53552ef61f6ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:46.450924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:46.542155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 992/1977] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:46.553808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.622057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d1f4b-0281-4354-8426-804bd70fdb95 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.626007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.626659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.626956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.627225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.643331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.645322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.656702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1978] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:46.776307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:46.788582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46cb0db-4b91-4a57-8b4c-80bb58b5e53e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:46.792906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:46.793744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:46.794088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:46.794425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:46.859257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:46.860676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d955259-b1ac-4ef3-8a6a-ed4313647fde service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:46.871005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 993/1979] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:49.082362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:49.083948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:49.105927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:49.152958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b4984-a605-4419-ad57-e0f2a5a1f292 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:49.157780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:49.158730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:49.159014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:49.159290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] PATH_INFO: `/v3/users/1d79d7ee1fe846ca8fa2bc82a88bb9ce` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:49.161072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d79d7ee1fe846ca8fa2bc82a88bb9ce)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:49.163064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:49.212603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d79d7ee1fe846ca8fa2bc82a88bb9ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.219920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d79d7ee1fe846ca8fa2bc82a88bb9ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.220628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d79d7ee1fe846ca8fa2bc82a88bb9ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.221241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d79d7ee1fe846ca8fa2bc82a88bb9ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.233259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d79d7ee1fe846ca8fa2bc82a88bb9ce'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.246085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1980] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:49 2022] DELETE /identity/v3/users/1d79d7ee1fe846ca8fa2bc82a88bb9ce => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:49.250303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:49.250805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:49.250895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:49.267943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:49.269430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:49.372604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 994/1981] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:49.382667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:49.384156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:49.443234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df06a6-bdd8-493a-858a-b80e76fdfc19 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:49.447323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:49.447985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:49.448296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:49.448621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:49.460383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:49.461834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:49.473057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1982] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:49.764732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:49.766303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:49.790245 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6017e3-faf3-45d1-84c7-4763a6ec6684 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:49.795424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:49.796350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:49.796634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:49.796909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] PATH_INFO: `/v3/projects/71fdda3a7c6e41ceb57a3cef30eec300` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:49.798107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fdda3a7c6e41ceb57a3cef30eec300)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:49.800122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:49.840740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:49.856875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fdda3a7c6e41ceb57a3cef30eec300', 'request_id': 'req-815c34c9-2e98-47d2-aa3a-51d68bb99038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.862664 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fdda3a7c6e41ceb57a3cef30eec300', 'request_id': 'req-815c34c9-2e98-47d2-aa3a-51d68bb99038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.871167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fdda3a7c6e41ceb57a3cef30eec300', 'request_id': 'req-815c34c9-2e98-47d2-aa3a-51d68bb99038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.882544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] The token cache is being invalidate because project 71fdda3a7c6e41ceb57a3cef30eec300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:49.882823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.883434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:49.885019 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 995/1983] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:49 2022] DELETE /identity/v3/projects/71fdda3a7c6e41ceb57a3cef30eec300 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:49.889002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:49.898681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad874fae-5182-4cbd-8d9e-f81eee7e0932 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:49.948226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:49.966056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:49.966912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:49.967209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:49.967461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] PATH_INFO: `/v3/users/19c0bf99faaf49a7b270e2e8696b2049` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:49.968751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c0bf99faaf49a7b270e2e8696b2049)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:49.970672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.018722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c0bf99faaf49a7b270e2e8696b2049'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.029230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c0bf99faaf49a7b270e2e8696b2049'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.029914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c0bf99faaf49a7b270e2e8696b2049'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.030637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c0bf99faaf49a7b270e2e8696b2049'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.044603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-392f69e4-f98f-4364-afde-3b149787c462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c0bf99faaf49a7b270e2e8696b2049'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.059089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1984] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:49 2022] DELETE /identity/v3/users/19c0bf99faaf49a7b270e2e8696b2049 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:50.345121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.346704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:50.360948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c34c9-2e98-47d2-aa3a-51d68bb99038 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.445501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.446652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.447006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.448312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] PATH_INFO: `/v3/projects/ca2378176a79476f81b2996c722d57db` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.449075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2378176a79476f81b2996c722d57db)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.451170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.514045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2378176a79476f81b2996c722d57db', 'request_id': 'req-66ad183f-805a-4293-9bad-c37ee3dde3a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.519695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2378176a79476f81b2996c722d57db', 'request_id': 'req-66ad183f-805a-4293-9bad-c37ee3dde3a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.529106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2378176a79476f81b2996c722d57db', 'request_id': 'req-66ad183f-805a-4293-9bad-c37ee3dde3a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.542117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] The token cache is being invalidate because project ca2378176a79476f81b2996c722d57db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:50.542406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.543057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-66ad183f-805a-4293-9bad-c37ee3dde3a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:50.544661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 996/1985] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:50 2022] DELETE /identity/v3/projects/ca2378176a79476f81b2996c722d57db => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:50.550247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.550760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.551098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.572059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:50.573288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:50.652176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1986] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:50.655826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.656316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.656713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.684203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:50.689290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:50.700539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 997/1987] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:50.702384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.714946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0af1f5-8353-4b05-9366-7fa33810f43a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.721252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.722629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.722912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.723189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.723928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:50.724274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.726192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.751285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1988] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:50.751936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.765137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f775f-1ce2-464c-8e63-5d285085b40f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.773123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.773912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.774114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.774357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.775048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.778245 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.818937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 998/1989] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:50 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:50.823191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.834659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fc94d-6b75-4303-8f83-dd22ccb12588 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.888677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:50.902529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.903210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.903484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.903751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.904472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.906405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.930919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1990] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:50 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:50.932195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.942907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5552ad-06d4-4f90-9b60-3ba59f9c1b0c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.947576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.948277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.948583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.948858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.949714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.951370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:50.963243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 999/1991] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:50.967322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:50.978902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea7aa05-fee7-4a23-983b-d8ead0f5423f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:50.991682 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:50.992486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:50.992760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:50.993031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:50.993767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:50.995412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0fa2f-d648-4e48-a014-cc088e8cdd6f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.006198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1992] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:51.010602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.020878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f2be8-2ae9-48bd-a839-1ee048408e51 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.025411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.026292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.026566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.027149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] PATH_INFO: `/v3/projects/04872ec1173546249e05d3be23af3513/users/810d7438d1d545c89226a0adb0f6bd2e/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.030168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:51.044344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04872ec1173546249e05d3be23af3513, user_id=810d7438d1d545c89226a0adb0f6bd2e, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.047572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.062957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1000/1993] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:51 2022] PUT /identity/v3/projects/04872ec1173546249e05d3be23af3513/users/810d7438d1d545c89226a0adb0f6bd2e/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:51.074639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.074922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.075198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.104394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] MFA Rules not processed for user `810d7438d1d545c89226a0adb0f6bd2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:51.105596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:51.170754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1994] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:51.175670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.249545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af43b87a-6c2c-4e22-8d1e-569e702b23e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.257032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.257695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.257989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.258239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.258950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.260898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.291279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1001/1995] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:51.295740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.307153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e54b78-91b5-4e9f-a388-51f589e03263 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.369178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.369884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.370285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.370560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.371269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.373294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.394495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/1996] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:51.398442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.409364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cac7a-500d-4610-b0ca-7c95572b7344 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.416316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.417564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.418461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.418834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.419582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.421958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.433589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1002/1997] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:51.437691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.447706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be436779-749f-4b7b-a3a3-3104e454083a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.452445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.453107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.453386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.453639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.454407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.456023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8961d0a-e257-4ced-b254-66dc4e906a5e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.466364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/1998] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:51.470598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.481921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb3d2d-efae-4790-87e4-494598de929b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.488177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.489186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.489476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.489752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] PATH_INFO: `/v3/projects/2e70d765fcd040789813771f098cfb15/users/4c0dfaa11efa427fb93dbf155da3df4e/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.510839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e70d765fcd040789813771f098cfb15, user_id=4c0dfaa11efa427fb93dbf155da3df4e, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.513838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.531697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1003/1999] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:51 2022] PUT /identity/v3/projects/2e70d765fcd040789813771f098cfb15/users/4c0dfaa11efa427fb93dbf155da3df4e/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:51.543661 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.545028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.545552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.588967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] MFA Rules not processed for user `4c0dfaa11efa427fb93dbf155da3df4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:51.591374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:51.619326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:51.666591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/2000] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:51.672345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.820863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96ab53-67db-40fd-826e-a04826495634 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:51.841679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.843268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:51.846992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:51.847614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:51.847989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:51.848480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:51.849211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:51.851157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:51.897318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1004/2001] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:51.898845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:51.946505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:51.962099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ff1b6-d231-401e-bd9c-38151bb703be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.036139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf59def-9ff4-4388-a1ed-f2c78e478a73 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.042639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.043706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.044182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.044606 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.045774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.049088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.056565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.057751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.057751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.058399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.060929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:52.085189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1005/2002] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:51 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:52.088728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.108417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1987710-6b1e-47e5-a2da-cf149b2ed240 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.113882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.116759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.116759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.116759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.117123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.120249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.137856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1006/2003] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:52.144502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.184957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d8b10-6cc7-45d8-9588-616d1dfbd6f9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.196064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.197640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.198177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.198648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] PATH_INFO: `/v3/projects/0d38479f753b42ce92291add5c75870b/users/b407691fef17431498a4b0a517d706b9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.212757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.214835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.231295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/2004] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:52.232688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d38479f753b42ce92291add5c75870b, user_id=b407691fef17431498a4b0a517d706b9, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.237319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.263213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1007/2005] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:52 2022] PUT /identity/v3/projects/0d38479f753b42ce92291add5c75870b/users/b407691fef17431498a4b0a517d706b9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:52.267051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.283933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17669b7-1aaa-4b33-99e1-28c152e2750f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.350845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:52.398921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.400081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.400549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.401032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.402312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.404621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5698f6a-5d94-4bc6-944f-0215b2ddbc46 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.419761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/2006] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:52.421240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.432521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef32f62-4072-4d3a-85cb-91d12a8fb1cc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.441697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.442655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.442875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.443142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] PATH_INFO: `/v3/projects/0d38479f753b42ce92291add5c75870b/users/b407691fef17431498a4b0a517d706b9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.445425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d38479f753b42ce92291add5c75870b, user_id=b407691fef17431498a4b0a517d706b9, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.448384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.482423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1008/2007] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:52 2022] PUT /identity/v3/projects/0d38479f753b42ce92291add5c75870b/users/b407691fef17431498a4b0a517d706b9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:52.495403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.495763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.496114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.543012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] MFA Rules not processed for user `b407691fef17431498a4b0a517d706b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:52.543553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:52.620904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/2008] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:52.643649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.648517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:52.747817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb610b4-7a9f-4032-a5e5-e98e074c0fed admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.753550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.754401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.754738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.755066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.767252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.769959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.782272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1009/2009] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:52.837929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.850164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c53b58-e4db-45ef-aa76-778212fd2d1c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:52.857701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:52.858906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:52.859023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:52.859290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:52.938360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:52.941504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:52.957220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:52.958767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0121766-0057-4ed3-846a-3dff93a87401 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:52.971683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/2010] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:53.017432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f8664-5469-4d60-b80b-6e95836dcbd4 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:53.023665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:53.024681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:53.024999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:53.025325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] PATH_INFO: `/v3/users/6f967a03794048e2b0023c4e42efbe18` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:53.027114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f967a03794048e2b0023c4e42efbe18)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:53.028999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:53.034178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:53.082959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f967a03794048e2b0023c4e42efbe18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.092670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f967a03794048e2b0023c4e42efbe18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.093196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f967a03794048e2b0023c4e42efbe18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.101279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f967a03794048e2b0023c4e42efbe18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.102038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f967a03794048e2b0023c4e42efbe18'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.122374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1010/2011] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:52 2022] DELETE /identity/v3/users/6f967a03794048e2b0023c4e42efbe18 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:53.127228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b32-995f-467d-9612-fb103361d992 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:53.127569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b32-995f-467d-9612-fb103361d992 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:53.127854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b32-995f-467d-9612-fb103361d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:53.160123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-499c1b32-995f-467d-9612-fb103361d992 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:53.161881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-499c1b32-995f-467d-9612-fb103361d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:53.218855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:53.238552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/2012] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:53.246541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:53.248864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:53.376592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963812f7-e855-4739-bba5-0a5d881708b6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:53.389309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:53.390334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:53.391540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:53.392155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:53.418996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:53.420607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:53.447210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1011/2013] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:53.806478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1b32-995f-467d-9612-fb103361d992 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:53.807871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-499c1b32-995f-467d-9612-fb103361d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:53.829745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1b32-995f-467d-9612-fb103361d992 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:53.835841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:53.836767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:53.837038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:53.837460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] PATH_INFO: `/v3/projects/6f55e272091b4ed8ab985f82e710aae8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:53.838666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f55e272091b4ed8ab985f82e710aae8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:53.841504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:53.898843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f55e272091b4ed8ab985f82e710aae8', 'request_id': 'req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.904938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f55e272091b4ed8ab985f82e710aae8', 'request_id': 'req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.913207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f55e272091b4ed8ab985f82e710aae8', 'request_id': 'req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.924510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] The token cache is being invalidate because project 6f55e272091b4ed8ab985f82e710aae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:53.924859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.925593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:53.927294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/2014] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:53 2022] DELETE /identity/v3/projects/6f55e272091b4ed8ab985f82e710aae8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:53.931306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:53.940773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad0cb-8183-45dc-8800-d8327ff1b334 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.000806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.001681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.001957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.002284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] PATH_INFO: `/v3/users/b3ef6f26ce2c43aaabe53552ef61f6ec` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.003468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ef6f26ce2c43aaabe53552ef61f6ec)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.005462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.036068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:54.055780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ef6f26ce2c43aaabe53552ef61f6ec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.063533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ef6f26ce2c43aaabe53552ef61f6ec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.064164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ef6f26ce2c43aaabe53552ef61f6ec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.071070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ef6f26ce2c43aaabe53552ef61f6ec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.071688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-55854042-3247-46a0-8d09-ba635b07f8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ef6f26ce2c43aaabe53552ef61f6ec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.089782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1012/2015] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:53 2022] DELETE /identity/v3/users/b3ef6f26ce2c43aaabe53552ef61f6ec => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:54.371512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.373180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:54.399880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e2bc3-3b9c-4aef-b6d7-37a16528d2f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.461983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.462863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.463139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.463409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] PATH_INFO: `/v3/projects/7eccc882f3274a61a7f437b5e9283213` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.464598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eccc882f3274a61a7f437b5e9283213)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.466528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.512915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eccc882f3274a61a7f437b5e9283213', 'request_id': 'req-1ba11fb2-988c-4df7-b95a-30c653162751'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.518524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eccc882f3274a61a7f437b5e9283213', 'request_id': 'req-1ba11fb2-988c-4df7-b95a-30c653162751'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.527797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eccc882f3274a61a7f437b5e9283213', 'request_id': 'req-1ba11fb2-988c-4df7-b95a-30c653162751'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.540019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] The token cache is being invalidate because project 7eccc882f3274a61a7f437b5e9283213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:54.540336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.541043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ba11fb2-988c-4df7-b95a-30c653162751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:54.542683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/2016] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:54 2022] DELETE /identity/v3/projects/7eccc882f3274a61a7f437b5e9283213 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:54.547734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.548215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.548368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.566397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:54.567694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:54.632770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1013/2017] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:54.641618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.641903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.642174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.652132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:54.664053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:54.665694 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:54.674850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/2018] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:54.678965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.688844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6bb4f-c621-4d1e-ae38-534b5326f5c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.693837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.694499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.694770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.695037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.695816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:54.696150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.697761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.710164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1014/2019] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:54.714298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.723725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6b1c3-50e8-4dfd-922f-b1cfc0fd6b5b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.731357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.732053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.732349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.732619 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.733306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.735292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.764203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/2020] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:54 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:54.768326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.779029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20c9dd-9368-4bf8-9bd6-3443e3ac90c1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.858232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.859225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.859508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.859775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.860511 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.863368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.889767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1015/2021] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:54 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:54.890939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.901417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045f4cc-fa49-4f80-903d-1b79bbd9f63f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.906375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.907056 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.907337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.907612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.908445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.910147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.921405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/2022] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:54.925549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.935697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a534603-cb46-4174-926a-98cadacb6222 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.940455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.941200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.941478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.941850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.942578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.944211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a6c52-a1c8-4d92-99fe-3b067bf3586f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:54.954446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1016/2023] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:54.958488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:54.968351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1e268-8bdb-49af-9cb1-41b17675162c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:54.973196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:54.974076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:54.974355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:54.974627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] PATH_INFO: `/v3/projects/79a07553bace4c8d9a56742802ce04cc/users/b5eb5033967a48d683e7eb6e6b59ac82/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:54.990532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a07553bace4c8d9a56742802ce04cc, user_id=b5eb5033967a48d683e7eb6e6b59ac82, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:54.993229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.007432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/2024] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:54 2022] PUT /identity/v3/projects/79a07553bace4c8d9a56742802ce04cc/users/b5eb5033967a48d683e7eb6e6b59ac82/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:55.017694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.017978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.018274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.038757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:55.047742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] MFA Rules not processed for user `b5eb5033967a48d683e7eb6e6b59ac82`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:55.049451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:55.116320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1017/2025] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:55.120861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.184492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb5cc2-2b45-4f7c-8389-d0fec8acf5ff admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.189666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.190327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.190607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.191112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.191798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.193750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.220802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/2026] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:55 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:55.226416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.238560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0835bac-2ffb-4d25-a9cb-d84e54e3df3b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.314051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.314739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.314984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.315282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.316251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.318196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.339350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1018/2027] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:55 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:55.343637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.354434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a224b-85ff-4e79-b4a1-fab3c9dfe673 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.359572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.360380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.360653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.360923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.361640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.363332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.374095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/2028] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:55.378420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.389006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31fc198-be32-41e8-86a8-8796c2ae0499 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.393846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.394949 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.395228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.395500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] PATH_INFO: `/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4/users/fa2bba89193e4aefb40683f113c78480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.410746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2f7d9cfa43444e8b03fdafeb61a3c4, user_id=fa2bba89193e4aefb40683f113c78480, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.413438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.430175 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1019/2029] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:55 2022] PUT /identity/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4/users/fa2bba89193e4aefb40683f113c78480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:55.433774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.444369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af432d-062a-4382-9420-3f721c38e032 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.506323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.507008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.507293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.507571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.508343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.509972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635b15d5-926d-48ef-97e7-dd51b6dce7eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.519749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/2030] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:55.524197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.534982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5626227-fc78-4a37-b977-beeabcf0a5fc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.539970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.540900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.541176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.541415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] PATH_INFO: `/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4/users/fa2bba89193e4aefb40683f113c78480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.543464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2f7d9cfa43444e8b03fdafeb61a3c4, user_id=fa2bba89193e4aefb40683f113c78480, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.546194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.556885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1020/2031] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:55 2022] PUT /identity/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4/users/fa2bba89193e4aefb40683f113c78480/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:55.568261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.568612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.568945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.594053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:55.603442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] MFA Rules not processed for user `fa2bba89193e4aefb40683f113c78480`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:55.605189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:55.681116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/2032] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:55.692871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:55.694755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:55.754810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef13da8-4751-408c-9a6c-dbee30b3f863 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:55.758524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:55.759192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:55.759473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:55.759749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:55.767474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:55.770884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:55.772480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc07f82f-17fc-4001-a38c-69211c544ab7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:55.785623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1021/2033] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:56.432528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:56.511248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e167c-004c-4c39-bb5b-77db86d3805f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:56.517507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:56.518893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:56.519242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:56.519582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] PATH_INFO: `/v3/users/b5eb5033967a48d683e7eb6e6b59ac82` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:56.521081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5eb5033967a48d683e7eb6e6b59ac82)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:56.524760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:56.591165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5eb5033967a48d683e7eb6e6b59ac82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:56.600662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5eb5033967a48d683e7eb6e6b59ac82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:56.601366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5eb5033967a48d683e7eb6e6b59ac82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:56.601999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5eb5033967a48d683e7eb6e6b59ac82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:56.615868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5eb5033967a48d683e7eb6e6b59ac82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:56.632062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/2034] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:56 2022] DELETE /identity/v3/users/b5eb5033967a48d683e7eb6e6b59ac82 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:56.636367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:56.636826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:56.636901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:56.661823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:56.670184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:56.757932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1022/2035] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:56.768281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:56.771448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:56.796775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:56.856573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324cf6b-d87c-42d4-bdfa-c7a1835769dd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:56.860542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:56.861199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:56.861755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:56.861755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:56.871665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:56.873171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:56.890334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2036] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:57.410362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:57.412116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:57.426434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f3e06-1c6f-4c71-91fe-e7d4038e3b73 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:57.434279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:57.435640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:57.436227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:57.436590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] PATH_INFO: `/v3/projects/79a07553bace4c8d9a56742802ce04cc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:57.438497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a07553bace4c8d9a56742802ce04cc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:57.442171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:57.501614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a07553bace4c8d9a56742802ce04cc', 'request_id': 'req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.508234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a07553bace4c8d9a56742802ce04cc', 'request_id': 'req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.516602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a07553bace4c8d9a56742802ce04cc', 'request_id': 'req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.528892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] The token cache is being invalidate because project 79a07553bace4c8d9a56742802ce04cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:57.529187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.529819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.531435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1023/2037] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:57 2022] DELETE /identity/v3/projects/79a07553bace4c8d9a56742802ce04cc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:57.535485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:57.545285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca3ca0-891f-4d19-83d7-f9b67fae18da service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:57.635847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:57.636782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:57.637064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:57.637338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] PATH_INFO: `/v3/users/fa2bba89193e4aefb40683f113c78480` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:57.638534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa2bba89193e4aefb40683f113c78480)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:57.640528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:57.688534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa2bba89193e4aefb40683f113c78480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.696531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa2bba89193e4aefb40683f113c78480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.697120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa2bba89193e4aefb40683f113c78480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.697776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa2bba89193e4aefb40683f113c78480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.710874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa2bba89193e4aefb40683f113c78480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:57.723320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2038] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:57 2022] DELETE /identity/v3/users/fa2bba89193e4aefb40683f113c78480 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:58.003486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.005380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.019769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a8fec-fc3e-44c3-b02f-52eb5ce4d6ad admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.093009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.094262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.094616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.094975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] PATH_INFO: `/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.096433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2f7d9cfa43444e8b03fdafeb61a3c4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.099781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.134389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.135808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.137136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:58.150370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63009d4b-53ac-4038-ba5c-535f7075a9e1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.155995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.156912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.157224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.157473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] PATH_INFO: `/v3/users/810d7438d1d545c89226a0adb0f6bd2e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.158668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=810d7438d1d545c89226a0adb0f6bd2e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.160730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.177472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2f7d9cfa43444e8b03fdafeb61a3c4', 'request_id': 'req-173688ef-77d1-4558-802c-98d0571472ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.184425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2f7d9cfa43444e8b03fdafeb61a3c4', 'request_id': 'req-173688ef-77d1-4558-802c-98d0571472ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.194615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2f7d9cfa43444e8b03fdafeb61a3c4', 'request_id': 'req-173688ef-77d1-4558-802c-98d0571472ac'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.205522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:58.206563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] The token cache is being invalidate because project 8b2f7d9cfa43444e8b03fdafeb61a3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:58.206933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.207708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-173688ef-77d1-4558-802c-98d0571472ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.213894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1024/2039] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:58 2022] DELETE /identity/v3/projects/8b2f7d9cfa43444e8b03fdafeb61a3c4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:58.214059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef48-35c7-48d9-b2ef-2487af7f3ec9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.214368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef48-35c7-48d9-b2ef-2487af7f3ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.214658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3ef48-35c7-48d9-b2ef-2487af7f3ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.224346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '810d7438d1d545c89226a0adb0f6bd2e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.233983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '810d7438d1d545c89226a0adb0f6bd2e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.234596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '810d7438d1d545c89226a0adb0f6bd2e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.234713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b0c3ef48-35c7-48d9-b2ef-2487af7f3ec9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:58.235304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '810d7438d1d545c89226a0adb0f6bd2e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.236379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0c3ef48-35c7-48d9-b2ef-2487af7f3ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.270447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-51adfcb2-e0b0-4ebc-b2c4-2559905fa4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '810d7438d1d545c89226a0adb0f6bd2e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:58.293545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2040] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:58 2022] DELETE /identity/v3/users/810d7438d1d545c89226a0adb0f6bd2e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:58.300729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.301187 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.301518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.328555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:58.329843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.352005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1025/2041] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:58.360977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.361448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.361545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.428891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:58.430199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.484047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2042] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:58.508136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.509711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:58.555091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1026/2043] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:38:58.561985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.578984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa193a2-a213-47eb-871d-d1a9e7c916e2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.589728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc43eff-630c-4121-b196-300e19cd8fc1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.598650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.600088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.600774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.601061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.601350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.602069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:38:58.602381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.603983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.604221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.604498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.616789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.625456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.627293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.632909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1027/2044] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:38:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:38:58.638102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.641037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2045] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:38:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:58.652855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefa345-02b3-4813-add2-255d5687ae22 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.657755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.658945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.658945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.659195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.660317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.662384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.692611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1028/2046] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:58.696326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.709550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6b84b-1dd8-403f-83fe-fb6855190327 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.765747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:58.780439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.781483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.782208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.782576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.783344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.786136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.809624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2047] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:58.813790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.827532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc25084-3b7f-4132-9a28-b217dd0de2e5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.833786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.834500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.834782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.835059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.836021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.837612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.848792 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1029/2048] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:58.852665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.862950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d3d44-839e-4f1d-a58b-c50acf80e596 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.868546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.869226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.869513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.869789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.870531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.872174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcf9bb-87cd-4e0f-a648-3e17ebd41ae1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.888928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:58.895387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2049] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:38:58.896877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:58.901169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010eacc-0428-4f32-b75f-81fed852b8aa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:58.907983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:58.909038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.909328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.909642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] PATH_INFO: `/v3/projects/cbf21fe9ce1c4b2cb901a9a34f574f29/users/b8ba5d97f3724c759520d5e006cd5cd9/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.929672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf21fe9ce1c4b2cb901a9a34f574f29, user_id=b8ba5d97f3724c759520d5e006cd5cd9, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:58.932017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:58.951948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1030/2050] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:58 2022] PUT /identity/v3/projects/cbf21fe9ce1c4b2cb901a9a34f574f29/users/b8ba5d97f3724c759520d5e006cd5cd9/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:38:58.962725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:58.963016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:58.963652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:58.991027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] MFA Rules not processed for user `b8ba5d97f3724c759520d5e006cd5cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:58.992280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.009327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.011402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.062400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2051] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:59.067058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.079054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75540dbe-aeac-4c8e-8d8e-73ab3b888189 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.084910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.085452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.085726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.086003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] PATH_INFO: `/v3/projects/04872ec1173546249e05d3be23af3513` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.087183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04872ec1173546249e05d3be23af3513)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.089168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.135828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f27980-07a1-441d-869e-d9f574174a8b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.140775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04872ec1173546249e05d3be23af3513', 'request_id': 'req-1db4707f-d60d-4929-8d56-f7cf81f49d59'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.146554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04872ec1173546249e05d3be23af3513', 'request_id': 'req-1db4707f-d60d-4929-8d56-f7cf81f49d59'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.155315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04872ec1173546249e05d3be23af3513', 'request_id': 'req-1db4707f-d60d-4929-8d56-f7cf81f49d59'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.166887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] The token cache is being invalidate because project 04872ec1173546249e05d3be23af3513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:38:59.167172 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.167825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.169421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1031/2052] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:59 2022] DELETE /identity/v3/projects/04872ec1173546249e05d3be23af3513 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:59.173424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.174735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.194379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.195044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.195325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.195601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.196550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.198492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.211532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:59.226292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2053] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:59 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:38:59.230515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.232382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.240296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4707f-d60d-4929-8d56-f7cf81f49d59 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.252468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:59.299036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.299294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78bf9a-3448-47f9-8186-8dc68091ad34 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.299914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.302627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.303398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] PATH_INFO: `/v3/users/4c0dfaa11efa427fb93dbf155da3df4e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.304121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.304668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c0dfaa11efa427fb93dbf155da3df4e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.304808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.305060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.305332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.306026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.306612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.307962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.328241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2054] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:59 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:59.332320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.343424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a79b14-9208-40dc-9170-4ab4c706be7b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.348629 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.349309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.349603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.349868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.350585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.352225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.357410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c0dfaa11efa427fb93dbf155da3df4e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.362793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2055] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:59.366803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c0dfaa11efa427fb93dbf155da3df4e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.367413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.367652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c0dfaa11efa427fb93dbf155da3df4e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.369364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.375132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c0dfaa11efa427fb93dbf155da3df4e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.375920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c0dfaa11efa427fb93dbf155da3df4e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:38:59.382337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dc129-3669-46ad-92ae-fff6253af042 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.387401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.388203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.388493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.388764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.389564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.391358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8250f3-c045-4c44-9fd6-9e2e0647f2fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.395125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1032/2056] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:38:59 2022] DELETE /identity/v3/users/4c0dfaa11efa427fb93dbf155da3df4e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:59.406118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2057] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:38:59.411434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.476145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514b9f9-6783-474a-86fb-a8c3d338e2d0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.482311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.483379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.483657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.483932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] PATH_INFO: `/v3/projects/95b6e6ba88f047b4b9a38f5273bdbe69/users/629cefba938c4b489342fed8bef770af/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.500919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b6e6ba88f047b4b9a38f5273bdbe69, user_id=629cefba938c4b489342fed8bef770af, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.505010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.520557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1033/2058] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:38:59 2022] PUT /identity/v3/projects/95b6e6ba88f047b4b9a38f5273bdbe69/users/629cefba938c4b489342fed8bef770af/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:38:59.531923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.532254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.533114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.563844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] MFA Rules not processed for user `629cefba938c4b489342fed8bef770af`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:38:59.565208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.649180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2059] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:38:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:38:59.653638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.696888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:59.719054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.721784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:59.776945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114500c-6309-468d-b799-cb9c45e7157f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.788876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.789889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.790415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.790876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.791838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.795142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.821284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:59.839225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1034/2060] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:38:59 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:59.843498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.845560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b893e5f-3d0f-4924-a7c4-6470efee1fb1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.916411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ee436-5240-4cac-9cdc-4e5ef54c13ab admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.923269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.924046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.924420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.924825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.925796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.927708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.932557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.933488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.934054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.934967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] PATH_INFO: `/v3/projects/2e70d765fcd040789813771f098cfb15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.936508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e70d765fcd040789813771f098cfb15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:59.938605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:59.963239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1035/2061] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:38:59 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:38:59.967896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:59.983668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265e8acb-415a-4be2-a11f-8e8d571feec0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:59.996733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:59.998686 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:59.998960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:59.999227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:59.999963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.002363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.021771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1036/2062] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:38:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:00.029455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e70d765fcd040789813771f098cfb15', 'request_id': 'req-be9ed922-52f0-4e78-9b10-071a7beee809'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.033475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.044807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e70d765fcd040789813771f098cfb15', 'request_id': 'req-be9ed922-52f0-4e78-9b10-071a7beee809'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.044968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78742710-e2c1-4405-88e4-7501c725d433 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.060552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e70d765fcd040789813771f098cfb15', 'request_id': 'req-be9ed922-52f0-4e78-9b10-071a7beee809'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.077425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] The token cache is being invalidate because project 2e70d765fcd040789813771f098cfb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:00.077798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.078519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.080224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2063] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:38:59 2022] DELETE /identity/v3/projects/2e70d765fcd040789813771f098cfb15 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:00.084738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.086198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.120823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.121645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.121933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.122214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] PATH_INFO: `/v3/projects/111248f8203c4647a65424b43d4d730a/users/f021d29b61ec4977a20ae5b23d6d9302/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.137771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] RBAC: Authorizing `identity:create_grant(project_id=111248f8203c4647a65424b43d4d730a, user_id=f021d29b61ec4977a20ae5b23d6d9302, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.140496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.156093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1037/2064] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:00 2022] PUT /identity/v3/projects/111248f8203c4647a65424b43d4d730a/users/f021d29b61ec4977a20ae5b23d6d9302/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:00.157631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed922-52f0-4e78-9b10-071a7beee809 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.160646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.162418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.172353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:00.197589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:00.224781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.225814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.226164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.226507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] PATH_INFO: `/v3/users/b407691fef17431498a4b0a517d706b9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.227792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b407691fef17431498a4b0a517d706b9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.229836 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.233693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555721e4-9d77-451a-a52a-ba7c6f1998be admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.238601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.239276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.239559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.239842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.240633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.242243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.255935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1038/2065] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:00.257139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.267962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1993-2bbb-402a-a064-b719c26248e6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.272839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.273751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.274029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.274298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] PATH_INFO: `/v3/projects/111248f8203c4647a65424b43d4d730a/users/f021d29b61ec4977a20ae5b23d6d9302/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.276281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=111248f8203c4647a65424b43d4d730a, user_id=f021d29b61ec4977a20ae5b23d6d9302, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.277114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b407691fef17431498a4b0a517d706b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.279229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1aee70-ef01-44cc-acab-3e94ce815ee8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.286089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b407691fef17431498a4b0a517d706b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.287454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b407691fef17431498a4b0a517d706b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.288235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b407691fef17431498a4b0a517d706b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.292164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1039/2066] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:00 2022] PUT /identity/v3/projects/111248f8203c4647a65424b43d4d730a/users/f021d29b61ec4977a20ae5b23d6d9302/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:00.302106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b407691fef17431498a4b0a517d706b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.302340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.302416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.302727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.319885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2067] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:00 2022] DELETE /identity/v3/users/b407691fef17431498a4b0a517d706b9 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:00.335100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] MFA Rules not processed for user `f021d29b61ec4977a20ae5b23d6d9302`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:00.336372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.410139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1040/2068] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:00.422436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.423890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.497418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09df2c-d68f-4823-b5a0-c46ca732ecc1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.501076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.501728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.502004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.502273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.504229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.566467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.568011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0331f-b91d-434e-a67b-0aef288b5661 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.577471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2069] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:00.605808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.607325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.710441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce9527-ff50-4eb4-8bad-f0a30170aa25 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:00.716128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:00.717003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.717283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.717558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] PATH_INFO: `/v3/projects/0d38479f753b42ce92291add5c75870b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.718774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d38479f753b42ce92291add5c75870b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:00.720766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:00.739724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:00.771785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d38479f753b42ce92291add5c75870b', 'request_id': 'req-49ba5179-439a-4f93-afb6-153a451a90eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.778110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d38479f753b42ce92291add5c75870b', 'request_id': 'req-49ba5179-439a-4f93-afb6-153a451a90eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.787086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d38479f753b42ce92291add5c75870b', 'request_id': 'req-49ba5179-439a-4f93-afb6-153a451a90eb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.799484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] The token cache is being invalidate because project 0d38479f753b42ce92291add5c75870b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:00.799777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.800574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-49ba5179-439a-4f93-afb6-153a451a90eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:00.802195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1041/2070] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:00 2022] DELETE /identity/v3/projects/0d38479f753b42ce92291add5c75870b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:00.808607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.809189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.809401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.833788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:00.835107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.845331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:00.940146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2071] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:00.944186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:00.944495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:00.944803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:00.965722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:00.966955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:00.975661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1042/2072] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:00.980120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:00.990571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46793f-e209-4018-be07-84deee60bb48 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.000209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.000882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.001163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.001461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.002190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:01.002497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.004543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.016475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2073] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:01.021484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.039134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e133ae-2ea1-4e2b-a6a8-e3eb24d965c3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.045028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.045858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.046195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.046475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.047181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.049199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.081363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1043/2074] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:01 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:01.083131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.110235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d457b-2d76-4417-9b4c-4ce778a55ec7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.183800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.184533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.184865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.185138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.185876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.187802 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.210288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2075] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:01 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:01.214176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.225744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64499cfb-881e-4db7-8c8d-a549d2724ad4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.230624 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.231355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.231704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.232057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.232872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.234545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.245390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1044/2076] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:01.249494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.260119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc2fdf-e93b-43eb-889d-afb267694a7d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.267637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.267637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.267637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.268023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.268641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.270407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d4baf8-cdad-4c07-b089-4105288924e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.281985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2077] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:01.286693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.297109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612d52c-47d4-4b6a-9a5b-c2ca1722fb08 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.302368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.303288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.303570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.303841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] PATH_INFO: `/v3/projects/1f6949b6e8884f3f9f431cf857efe1d0/users/1c25430062e64719a57a4c94d15b40d5/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.320644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6949b6e8884f3f9f431cf857efe1d0, user_id=1c25430062e64719a57a4c94d15b40d5, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.323295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.338721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1045/2078] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:01 2022] PUT /identity/v3/projects/1f6949b6e8884f3f9f431cf857efe1d0/users/1c25430062e64719a57a4c94d15b40d5/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:01.350632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.350902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.351175 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.380978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] MFA Rules not processed for user `1c25430062e64719a57a4c94d15b40d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:01.382213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:01.459777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2079] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:01.474743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:01.477565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:01.540550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953273f3-4f8b-4aea-acf3-bc5e22a0f492 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:01.544810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:01.545471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:01.545749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:01.546015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:01.555307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:01.556789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:01.566083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1046/2080] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:03.206334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:03.207971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:03.216560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:03.221127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a4c55-8fe0-471c-8cf2-e6fe54dbf913 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:03.227348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:03.227348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:03.227348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:03.227885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:03.230005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:03.299882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:03.300693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:03.315767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2081] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:04.235800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:04.246017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62c7f2-c019-4740-9276-e70e7b72cf11 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:04.249933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:04.250848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:04.251288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:04.251715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:04.254312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:04.317714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:04.349382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:04.350994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7545-fd66-4689-8a27-5cb205b515fd service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:04.362843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1047/2082] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:11.570112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:11.643491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b1a2c-a54f-4de7-ad39-2305116ba23c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:11.654135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:11.655270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:11.655680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:11.656318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] PATH_INFO: `/v3/users/1c25430062e64719a57a4c94d15b40d5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:11.658152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c25430062e64719a57a4c94d15b40d5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:11.660337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:11.770642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c25430062e64719a57a4c94d15b40d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:11.783663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c25430062e64719a57a4c94d15b40d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:11.784609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c25430062e64719a57a4c94d15b40d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:11.785367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c25430062e64719a57a4c94d15b40d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:11.804460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c25430062e64719a57a4c94d15b40d5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:11.823433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2083] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:11 2022] DELETE /identity/v3/users/1c25430062e64719a57a4c94d15b40d5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:11.824819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:11.825247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:11.825521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:11.846815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:11.849202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:11.935001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1048/2084] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:11.945161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:11.946737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:11.994647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:12.050335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4df923-a4bc-4b0e-932c-236f8bf5a58f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:12.055411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:12.058274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.058696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.060401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.077546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:12.079079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faba7e-9c87-4280-a4d9-fe107ec6c177 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:12.091477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2085] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:12.588813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:12.590433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:12.603741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd60f97-ba9f-4268-88b9-ed5b618325ec None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:12.608874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:12.611055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.611158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.611517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] PATH_INFO: `/v3/projects/1f6949b6e8884f3f9f431cf857efe1d0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.612729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6949b6e8884f3f9f431cf857efe1d0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:12.614715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:12.664644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6949b6e8884f3f9f431cf857efe1d0', 'request_id': 'req-4f12209a-d841-47db-b1da-8e225ff8f7dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:12.671148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6949b6e8884f3f9f431cf857efe1d0', 'request_id': 'req-4f12209a-d841-47db-b1da-8e225ff8f7dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:12.680086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6949b6e8884f3f9f431cf857efe1d0', 'request_id': 'req-4f12209a-d841-47db-b1da-8e225ff8f7dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:12.691451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] The token cache is being invalidate because project 1f6949b6e8884f3f9f431cf857efe1d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:12.691794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:12.692455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f12209a-d841-47db-b1da-8e225ff8f7dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:12.694063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1049/2086] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:12 2022] DELETE /identity/v3/projects/1f6949b6e8884f3f9f431cf857efe1d0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:12.699314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.699780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.699866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.718671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:12.719902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:12.780767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2087] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:12.789558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.789840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.790176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.809410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:12.810632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:12.818866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1050/2088] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:12.823123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:12.834119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824197a2-a85c-4f4a-bba9-f26a6c411473 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:12.838969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:12.839815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.840182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.840521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.841318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:12.841730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:12.843404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:12.856856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2089] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:12.861411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:12.875719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960e635-16ad-4869-be90-80edc76ae1b7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:12.887345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:12.888020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:12.888323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:12.888627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:12.889323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:12.891256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:12.928014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1051/2090] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:12 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:12.929100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:12.938960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26f5fb-d12d-4d69-813f-04cec2cc73da admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.004301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.005327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.005620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.006026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.006716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.008724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.033269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2091] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:12 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:13.034274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.045907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397845f-c8fb-48ff-a857-a814e5a06336 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.051453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.052260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.052537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.052804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.053536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.055141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.066102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1052/2092] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:13.071032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.081950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291f021-1f75-47ab-8c3e-488305151f48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.086805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.087471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.087752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.088074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.088782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.090420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033595b-64b9-46f9-832f-168e546353e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.101235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2093] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:13.106735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.119066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d635650-ada0-4f7a-80a3-2af203b3937f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.124317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.125193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.125471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.125755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] PATH_INFO: `/v3/projects/6dfc8a42259240f091a60df654a51129/users/30756eb5a7ac4f2683c9818ebde03553/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.141592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dfc8a42259240f091a60df654a51129, user_id=30756eb5a7ac4f2683c9818ebde03553, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.143749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.153629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:13.171317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1053/2094] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:13 2022] PUT /identity/v3/projects/6dfc8a42259240f091a60df654a51129/users/30756eb5a7ac4f2683c9818ebde03553/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:13.172582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95935d-082e-4186-9764-09941bd6312a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.172868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95935d-082e-4186-9764-09941bd6312a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.173145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95935d-082e-4186-9764-09941bd6312a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.203922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f95935d-082e-4186-9764-09941bd6312a None None] MFA Rules not processed for user `30756eb5a7ac4f2683c9818ebde03553`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:13.205478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f95935d-082e-4186-9764-09941bd6312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.214120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.216461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.228362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:13.286256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2095] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:13 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:13.303238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95935d-082e-4186-9764-09941bd6312a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.305288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f95935d-082e-4186-9764-09941bd6312a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.306762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b6642-398d-4b9c-a295-24ebcd2b7a66 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.312292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.313922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.314304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.314678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] PATH_INFO: `/v3/users/6ec41eb194664352860938f771d9e356` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.316166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec41eb194664352860938f771d9e356)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.318373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.378624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95935d-082e-4186-9764-09941bd6312a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.378801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec41eb194664352860938f771d9e356'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:13.382282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.382931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.383242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.383531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.386998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec41eb194664352860938f771d9e356'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:13.387765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec41eb194664352860938f771d9e356'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:13.392441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.393885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205815aa-7b96-4e56-92c7-14f1c30a2714 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.394956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec41eb194664352860938f771d9e356'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:13.395683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec41eb194664352860938f771d9e356'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:13.403943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2096] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:13 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:13.413929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1054/2097] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:13 2022] DELETE /identity/v3/users/6ec41eb194664352860938f771d9e356 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:13.418371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.418730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.419127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.437387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:13.438512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.500636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2098] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:13.510427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.512016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.571642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73232-0ae2-470e-863c-9623be5a5364 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.575290 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.575934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.576256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.576528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.585945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.587253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.598288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1055/2099] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:13.910823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:13.913879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:13.933557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5eaa8f-87a0-4a7e-aeaf-7aaa3f4b114c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:13.938918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:13.939803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:13.940140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:13.940425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] PATH_INFO: `/v3/projects/065eb0050b8448889c43470fd59f0549` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:13.941622 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] RBAC: Authorizing `identity:delete_project(project_id=065eb0050b8448889c43470fd59f0549)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:13.943747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:13.996650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065eb0050b8448889c43470fd59f0549', 'request_id': 'req-a7385a3c-b30d-4df0-8f14-f45307ed0def'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.002590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065eb0050b8448889c43470fd59f0549', 'request_id': 'req-a7385a3c-b30d-4df0-8f14-f45307ed0def'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.004405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:14.011124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065eb0050b8448889c43470fd59f0549', 'request_id': 'req-a7385a3c-b30d-4df0-8f14-f45307ed0def'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.023407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] The token cache is being invalidate because project 065eb0050b8448889c43470fd59f0549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:14.023741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.024396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.026342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2100] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:13 2022] DELETE /identity/v3/projects/065eb0050b8448889c43470fd59f0549 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:14.030306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:14.040997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c2097-ff97-418e-930e-d2d194f81f71 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:14.058214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:14.100808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:14.101691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.101971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.102268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] PATH_INFO: `/v3/users/ef3ac2d7fb7d48b38f929ce87c988945` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.103563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3ac2d7fb7d48b38f929ce87c988945)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:14.105578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:14.152505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3ac2d7fb7d48b38f929ce87c988945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.160198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3ac2d7fb7d48b38f929ce87c988945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.160886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3ac2d7fb7d48b38f929ce87c988945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.167923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3ac2d7fb7d48b38f929ce87c988945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.168678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ff22c6af-37ff-4540-88bc-de3748962c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3ac2d7fb7d48b38f929ce87c988945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.187613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1056/2101] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:14 2022] DELETE /identity/v3/users/ef3ac2d7fb7d48b38f929ce87c988945 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:14.551609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:14.553226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:14.566185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7385a3c-b30d-4df0-8f14-f45307ed0def admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:14.633861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:14.634768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.635050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.635324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] PATH_INFO: `/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.636580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1cd213aa4284b048c58d3cef0bf5ec2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:14.638552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:14.686842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1cd213aa4284b048c58d3cef0bf5ec2', 'request_id': 'req-47935f9d-d575-4112-b0af-e06b967230d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.692945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1cd213aa4284b048c58d3cef0bf5ec2', 'request_id': 'req-47935f9d-d575-4112-b0af-e06b967230d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.702981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1cd213aa4284b048c58d3cef0bf5ec2', 'request_id': 'req-47935f9d-d575-4112-b0af-e06b967230d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.714299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] The token cache is being invalidate because project e1cd213aa4284b048c58d3cef0bf5ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:14.714589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.715206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47935f9d-d575-4112-b0af-e06b967230d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:14.716797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2102] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:14 2022] DELETE /identity/v3/projects/e1cd213aa4284b048c58d3cef0bf5ec2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:14.721684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.721979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.722258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.740084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:14.741223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:14.805847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1057/2103] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:14.813844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.814131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.814406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.834461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:14.835702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:14.844664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2104] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:14.849023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:14.859119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6d8a0-31c9-4da2-8b9f-584300d68151 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:14.864436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:14.865087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.865388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.865653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.866378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:14.866676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:14.868320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:14.880651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1058/2105] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:14.885072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:14.895804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2f6f6-db90-4391-b30a-f08c548e97fc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:14.902434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:14.903097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:14.903374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:14.903640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:14.904362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:14.906278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:14.937647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2106] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:14 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:14.939434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:14.949897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8449d-6ca2-4c11-9e5a-6dd9c443b637 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:14.974326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:15.008486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.009080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.009355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.009625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.010333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.012298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.033264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1059/2107] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:14 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:15.037400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.047635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f415cbb-f80d-4744-ac4d-925a9b3c32f7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.052895 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.053561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.053841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.054117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.054853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.056550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.067237 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2108] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:15.073151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.084639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76677b0-097d-44dd-bc80-f1629b9f63e8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.089787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.090454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.090724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.091129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.091865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.093482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207e1d4-1a03-41b0-8e3d-f2159b1825df admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.104426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1060/2109] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:15.108824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.119213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b869f-b20c-4396-99a4-298329dd79c8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.123926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.124857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.125139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.125416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] PATH_INFO: `/v3/projects/c1ced12d86a64f9087a299181cc33cf4/users/6837da4fb5d34384a53cb954d4a46cab/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.140739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ced12d86a64f9087a299181cc33cf4, user_id=6837da4fb5d34384a53cb954d4a46cab, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.143432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.157857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2110] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:15 2022] PUT /identity/v3/projects/c1ced12d86a64f9087a299181cc33cf4/users/6837da4fb5d34384a53cb954d4a46cab/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:15.169075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.169378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.169651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.198235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] MFA Rules not processed for user `6837da4fb5d34384a53cb954d4a46cab`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:15.199451 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:15.277897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1061/2111] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:15.282348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.300371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:15.350350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159aa65c-09ff-4337-8356-6d8e7f08ca16 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.355318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.355982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.356288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.356560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.357249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.359165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.386450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2112] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:15.391300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.404750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d5940-738a-44b7-8fb3-10c915f0d780 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.466693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.467378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.467659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.467926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.468638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.470569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.494714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1062/2113] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:15 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:15.506053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.510386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a67b1-8b82-47be-a0ab-86106e4d4dcf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.515590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.516298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.516574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.516856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.517600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.519219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.531532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2114] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:15.535969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.546137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303587d6-a0bb-4cff-89ef-d5edbefcae5e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.551745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.552759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.553037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.553306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] PATH_INFO: `/v3/projects/5b15f83b15014f388074f9284f7480cc/users/f3c99582a9be4ccd9971eaa5fc8c0356/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.568874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b15f83b15014f388074f9284f7480cc, user_id=f3c99582a9be4ccd9971eaa5fc8c0356, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.571554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.581370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:15.589906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1063/2115] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:15 2022] PUT /identity/v3/projects/5b15f83b15014f388074f9284f7480cc/users/f3c99582a9be4ccd9971eaa5fc8c0356/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:15.593776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.606432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f575a-a05a-4465-bc56-647c0db8687d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.677362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.678128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.678405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.678676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.679409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.681060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b78e2-9de7-4e73-a97e-313d540d0cc1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.703045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2116] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:15.704300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.715973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705fb29-e254-4062-b1f1-38d2c6edf2ce admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.720842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.721744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.722028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.722304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] PATH_INFO: `/v3/projects/5b15f83b15014f388074f9284f7480cc/users/f3c99582a9be4ccd9971eaa5fc8c0356/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.725024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b15f83b15014f388074f9284f7480cc, user_id=f3c99582a9be4ccd9971eaa5fc8c0356, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.727702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.738776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1064/2117] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:15 2022] PUT /identity/v3/projects/5b15f83b15014f388074f9284f7480cc/users/f3c99582a9be4ccd9971eaa5fc8c0356/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:15.749456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.749774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.750727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.780602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] MFA Rules not processed for user `f3c99582a9be4ccd9971eaa5fc8c0356`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:15.781900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:15.856233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2118] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:15.867539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:15.869543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:15.929148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7678d1-94ae-4e09-82e9-59322473b9e7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:15.933408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:15.934107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:15.934386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:15.934656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:15.945378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:15.946846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a493-09c6-4f00-b15d-fd0b5bf423a5 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:15.958882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1065/2119] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:16.101828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:16.111649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:16.169505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738ff86-d11a-482a-8b4b-0e08f63986f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:16.174514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:16.175446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:16.175707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:16.176114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] PATH_INFO: `/v3/users/6837da4fb5d34384a53cb954d4a46cab` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:16.177470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6837da4fb5d34384a53cb954d4a46cab)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:16.179445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:16.230536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6837da4fb5d34384a53cb954d4a46cab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.237876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6837da4fb5d34384a53cb954d4a46cab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.238841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6837da4fb5d34384a53cb954d4a46cab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.239680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6837da4fb5d34384a53cb954d4a46cab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.255970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6837da4fb5d34384a53cb954d4a46cab'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.271659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2120] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:16 2022] DELETE /identity/v3/users/6837da4fb5d34384a53cb954d4a46cab => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:16.277026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:16.277638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:16.277638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:16.300551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:16.301850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:16.373767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1066/2121] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:16.384987 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:16.386463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:16.454019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb52dac-27d9-47b1-b517-c1625bf1fd5a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:16.460612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:16.461275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:16.461556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:16.461825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:16.473160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:16.474466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:16.484676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2122] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:16.768209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:16.770129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:16.806932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44201ea7-49cb-47f5-bd78-de2bfe73b755 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:16.813252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:16.814519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:16.814991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:16.815330 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] PATH_INFO: `/v3/projects/c1ced12d86a64f9087a299181cc33cf4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:16.816694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ced12d86a64f9087a299181cc33cf4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:16.818759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:16.910197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ced12d86a64f9087a299181cc33cf4', 'request_id': 'req-957d84dc-dddd-401e-b47f-32d60f6242a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.916442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ced12d86a64f9087a299181cc33cf4', 'request_id': 'req-957d84dc-dddd-401e-b47f-32d60f6242a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.924542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ced12d86a64f9087a299181cc33cf4', 'request_id': 'req-957d84dc-dddd-401e-b47f-32d60f6242a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.936736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] The token cache is being invalidate because project c1ced12d86a64f9087a299181cc33cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:16.937376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.937678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:16.939438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1067/2123] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:16 2022] DELETE /identity/v3/projects/c1ced12d86a64f9087a299181cc33cf4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:16.943555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:16.954416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6252c-5f02-46af-ae19-64161abb5393 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:16.986971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:17.017149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.018481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.018481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.018658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] PATH_INFO: `/v3/users/f3c99582a9be4ccd9971eaa5fc8c0356` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.019719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3c99582a9be4ccd9971eaa5fc8c0356)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.021846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.076656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3c99582a9be4ccd9971eaa5fc8c0356'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.084008 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3c99582a9be4ccd9971eaa5fc8c0356'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.084727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3c99582a9be4ccd9971eaa5fc8c0356'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.085314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3c99582a9be4ccd9971eaa5fc8c0356'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.096979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6710ea55-cf24-4b3d-8883-8b4a5138e26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3c99582a9be4ccd9971eaa5fc8c0356'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.109085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2124] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:16 2022] DELETE /identity/v3/users/f3c99582a9be4ccd9971eaa5fc8c0356 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:17.388437 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.390495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:17.404675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d84dc-dddd-401e-b47f-32d60f6242a8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.414909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:17.494222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.495265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.495590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.495986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] PATH_INFO: `/v3/projects/5b15f83b15014f388074f9284f7480cc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.497679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b15f83b15014f388074f9284f7480cc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.499661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.562585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b15f83b15014f388074f9284f7480cc', 'request_id': 'req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.570787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b15f83b15014f388074f9284f7480cc', 'request_id': 'req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.586222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b15f83b15014f388074f9284f7480cc', 'request_id': 'req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.604573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] The token cache is being invalidate because project 5b15f83b15014f388074f9284f7480cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:17.605731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.607037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aecd15e0-4c95-4fbc-bd73-815c6d16f8e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:17.617012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1068/2125] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:17 2022] DELETE /identity/v3/projects/5b15f83b15014f388074f9284f7480cc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:17.618043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.618330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.618606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.643649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:17.645394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:17.716434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2126] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:17.718040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.718326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.718629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.737299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:17.738578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:17.747484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1069/2127] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:17.751741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.762898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6cab3-f565-47e6-b88e-d2e286fc7948 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.767697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.768397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.768674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.768945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.769885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:17.770210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.771813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.788416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2128] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:17.792878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.803657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c81cc19-f8d0-4336-aa2d-725d6e04af7c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.809231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.809870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.810182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.810428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.811163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.813191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.845749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1070/2129] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:17 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:17.853737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.862379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a9e07-8e06-4bb2-a6f8-de2d012360ab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.898092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:17.933420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.934104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.935351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.935351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.935351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.937710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.961921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2130] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:17 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:17.966190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.972376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:17.978242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:17.982372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d70e0e2-3b47-4f49-b2d9-959064a72ecf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.987940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:17.988790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:17.989105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:17.989386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:17.990153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:17.991992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a59a20-c10d-4875-9061-c6bd829b8d53 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:17.992292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:17.999000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.000492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.000839 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.001171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] PATH_INFO: `/v3/users/b8ba5d97f3724c759520d5e006cd5cd9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.002730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ba5d97f3724c759520d5e006cd5cd9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.005406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1071/2131] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:18.006372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.008991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.020335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9dac44-083f-41bb-acd4-95b34230cab2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.028741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.029419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.029689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.030106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.030835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.033070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.049806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1072/2132] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:18.054106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.066374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb276bac-ef6d-4c4c-b548-2f6258ab4df8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.071025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ba5d97f3724c759520d5e006cd5cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.071713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.072675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.073039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.073384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] PATH_INFO: `/v3/projects/2f2e7ed467ef46d69cd491d92fce609c/users/dc15c8e6275140adb222953c5f55ed16/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.080432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ba5d97f3724c759520d5e006cd5cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.081182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ba5d97f3724c759520d5e006cd5cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.081949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ba5d97f3724c759520d5e006cd5cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.098160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f2e7ed467ef46d69cd491d92fce609c, user_id=dc15c8e6275140adb222953c5f55ed16, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.101077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60ba61-3db0-4ccb-9e7d-c059b970aa7a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.106620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-95215d9e-8921-4967-ae20-c9d909169c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ba5d97f3724c759520d5e006cd5cd9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.121944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2133] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:17 2022] DELETE /identity/v3/users/b8ba5d97f3724c759520d5e006cd5cd9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:18.126331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.126617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.126892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.128113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1073/2134] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:18 2022] PUT /identity/v3/projects/2f2e7ed467ef46d69cd491d92fce609c/users/dc15c8e6275140adb222953c5f55ed16/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:18.132480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.132876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.133151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.145436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:18.146697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.162121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] MFA Rules not processed for user `dc15c8e6275140adb222953c5f55ed16`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:18.163463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.183252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:18.211493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2135] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:18.221271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.222609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.236155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1074/2136] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:18.240690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.241977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.255029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e00987-671e-4c76-95f7-8eda257269e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.259771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.260456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.260737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.261009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.261700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.263614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.267630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:18.287098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb67a6-34b0-436a-9b2f-94f8d007674e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.290821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.291504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.291772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.292057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.292984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1075/2137] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:18 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:18.296728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.302182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.303649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.314984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2138] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:18.373113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56695e-f888-4eb6-b926-7a84ca1606ca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.378822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.379770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.380068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.380343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.381032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.383192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.404824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1076/2139] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:18 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:18.408897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.419875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575b929-4b6d-4e3e-a97a-89c1b92239c6 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.425140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.425810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.426436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.426729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.427440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.429156 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.443169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2140] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:18.444190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.454602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd91b1-521e-497e-bb0e-f57b02813bfa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.459404 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.460401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.460765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.461098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] PATH_INFO: `/v3/projects/65db607bd69745789d47ef270f233a4e/users/af958351178a4ace9f2da389d90b74e4/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.476945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=65db607bd69745789d47ef270f233a4e, user_id=af958351178a4ace9f2da389d90b74e4, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.479555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.498108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1077/2141] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:18 2022] PUT /identity/v3/projects/65db607bd69745789d47ef270f233a4e/users/af958351178a4ace9f2da389d90b74e4/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:18.502084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.512074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff778f-9ece-4a66-af41-9bce0f534471 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.569654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.570326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.570603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.570872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.571589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.573256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.583992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2142] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:18.587996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.599762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53e81b-f645-4626-b96c-4b4170be05cf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.600853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.602625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.604930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.605885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.606236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.607255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] PATH_INFO: `/v3/projects/65db607bd69745789d47ef270f233a4e/users/af958351178a4ace9f2da389d90b74e4/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.609935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=65db607bd69745789d47ef270f233a4e, user_id=af958351178a4ace9f2da389d90b74e4, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.612892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1ad1d5-71ab-4baf-9484-c3df7bd7a7fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.623315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af1c4f-b6ef-4bab-8201-415e91a882d7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.629169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.630304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.630533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.630789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] PATH_INFO: `/v3/projects/cbf21fe9ce1c4b2cb901a9a34f574f29` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.632289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf21fe9ce1c4b2cb901a9a34f574f29)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.634435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.637346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1078/2143] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:18 2022] PUT /identity/v3/projects/65db607bd69745789d47ef270f233a4e/users/af958351178a4ace9f2da389d90b74e4/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:18.648484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.648806 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.649041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.693214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] MFA Rules not processed for user `af958351178a4ace9f2da389d90b74e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:18.694331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.702852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf21fe9ce1c4b2cb901a9a34f574f29', 'request_id': 'req-645bf22a-36b7-4fad-a7b7-db821222d89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.709083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf21fe9ce1c4b2cb901a9a34f574f29', 'request_id': 'req-645bf22a-36b7-4fad-a7b7-db821222d89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.721390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf21fe9ce1c4b2cb901a9a34f574f29', 'request_id': 'req-645bf22a-36b7-4fad-a7b7-db821222d89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.741327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] The token cache is being invalidate because project cbf21fe9ce1c4b2cb901a9a34f574f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:18.741641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.744430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.746336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:18.746491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2144] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:18 2022] DELETE /identity/v3/projects/cbf21fe9ce1c4b2cb901a9a34f574f29 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:18.750708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.752150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.790839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1079/2145] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:18.801247 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:18.802776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:18.838392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:18.854529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645bf22a-36b7-4fad-a7b7-db821222d89c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.862026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.863716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.864245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.864693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] PATH_INFO: `/v3/users/629cefba938c4b489342fed8bef770af` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.866121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629cefba938c4b489342fed8bef770af)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.869127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.895226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e4381-b81b-45b1-8008-55ac03f44ef1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:18.901569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:18.902355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:18.902762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:18.903033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:18.913389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:18.915136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:18.921333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629cefba938c4b489342fed8bef770af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.924937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1080/2146] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:18.930292 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629cefba938c4b489342fed8bef770af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.930884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629cefba938c4b489342fed8bef770af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.931503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629cefba938c4b489342fed8bef770af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.944690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629cefba938c4b489342fed8bef770af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:18.957248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2147] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:18 2022] DELETE /identity/v3/users/629cefba938c4b489342fed8bef770af => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:19.369789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:19.388560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270f4c5-2089-4a38-8c4f-0ab125cc6d69 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:19.396505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:19.397958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:19.497859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:19.498741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:19.499025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:19.499301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] PATH_INFO: `/v3/projects/95b6e6ba88f047b4b9a38f5273bdbe69` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:19.500582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b6e6ba88f047b4b9a38f5273bdbe69)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:19.501353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f77cda-df3c-4303-aac0-2f41ee061163 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:19.502542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:19.506219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:19.507096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:19.507375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:19.507641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] PATH_INFO: `/v3/users/dc15c8e6275140adb222953c5f55ed16` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:19.508890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc15c8e6275140adb222953c5f55ed16)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:19.510847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:19.554063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b6e6ba88f047b4b9a38f5273bdbe69', 'request_id': 'req-6965955b-3398-47b6-92ff-0af29343dad6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.559797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b6e6ba88f047b4b9a38f5273bdbe69', 'request_id': 'req-6965955b-3398-47b6-92ff-0af29343dad6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.560543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc15c8e6275140adb222953c5f55ed16'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.569021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b6e6ba88f047b4b9a38f5273bdbe69', 'request_id': 'req-6965955b-3398-47b6-92ff-0af29343dad6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.571036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc15c8e6275140adb222953c5f55ed16'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.571760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc15c8e6275140adb222953c5f55ed16'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.572448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc15c8e6275140adb222953c5f55ed16'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.583719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] The token cache is being invalidate because project 95b6e6ba88f047b4b9a38f5273bdbe69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:19.584062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.584774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.586726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1081/2148] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:19 2022] DELETE /identity/v3/projects/95b6e6ba88f047b4b9a38f5273bdbe69 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:19.588326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-df097f0b-e55f-463c-8c9c-3a32384059ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc15c8e6275140adb222953c5f55ed16'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.589620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:19.590460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:19.592184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:19.603131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:19.604248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2149] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:19 2022] DELETE /identity/v3/users/dc15c8e6275140adb222953c5f55ed16 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:19.608464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:19.608763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:19.609043 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:19.627788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:19.629106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:19.661921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965955b-3398-47b6-92ff-0af29343dad6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:19.698429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2150] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:19.708232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:19.709960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:19.720246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:19.721011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:19.721353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:19.721626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] PATH_INFO: `/v3/users/f021d29b61ec4977a20ae5b23d6d9302` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:19.722907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f021d29b61ec4977a20ae5b23d6d9302)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:19.725020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:19.774220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b31a4-7a7b-4ea5-95f1-1a015af2126c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:19.778806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:19.779490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:19.779770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:19.780083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:19.793943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f021d29b61ec4977a20ae5b23d6d9302'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.801225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:19.803328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:19.812712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f021d29b61ec4977a20ae5b23d6d9302'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.813508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f021d29b61ec4977a20ae5b23d6d9302'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.824623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f021d29b61ec4977a20ae5b23d6d9302'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.826183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f021d29b61ec4977a20ae5b23d6d9302'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:19.829377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2151] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:19.846861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1082/2152] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:19 2022] DELETE /identity/v3/users/f021d29b61ec4977a20ae5b23d6d9302 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:20.145630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.147334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.162800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48802-2d09-405e-8271-01e4ec782410 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.226851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.228157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.228533 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.229036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] PATH_INFO: `/v3/projects/2f2e7ed467ef46d69cd491d92fce609c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.230459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f2e7ed467ef46d69cd491d92fce609c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.240386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.289810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:20.297842 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.299743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.305676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f2e7ed467ef46d69cd491d92fce609c', 'request_id': 'req-42e6335b-25d2-48e0-9730-75d8e515ff4a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.312212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f2e7ed467ef46d69cd491d92fce609c', 'request_id': 'req-42e6335b-25d2-48e0-9730-75d8e515ff4a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.314521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8dfe3-c694-4db3-b223-4f7f81127534 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.324713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f2e7ed467ef46d69cd491d92fce609c', 'request_id': 'req-42e6335b-25d2-48e0-9730-75d8e515ff4a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.349288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] The token cache is being invalidate because project 2f2e7ed467ef46d69cd491d92fce609c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:20.349586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.350219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.354871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2153] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:20 2022] DELETE /identity/v3/projects/2f2e7ed467ef46d69cd491d92fce609c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:20.355776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.357274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.362764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:20.398314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.399132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.399405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.399671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] PATH_INFO: `/v3/users/afeb858ab0db49c08bb1f7cfb4ce8942` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.401207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afeb858ab0db49c08bb1f7cfb4ce8942)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.403134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.429177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6335b-25d2-48e0-9730-75d8e515ff4a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.434912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.435793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.436102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.436375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] PATH_INFO: `/v3/projects/111248f8203c4647a65424b43d4d730a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.437545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=111248f8203c4647a65424b43d4d730a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.439517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.452790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afeb858ab0db49c08bb1f7cfb4ce8942'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.461785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afeb858ab0db49c08bb1f7cfb4ce8942'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.462588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afeb858ab0db49c08bb1f7cfb4ce8942'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.470509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afeb858ab0db49c08bb1f7cfb4ce8942'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.471226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afeb858ab0db49c08bb1f7cfb4ce8942'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.495075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1083/2154] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:20 2022] DELETE /identity/v3/users/afeb858ab0db49c08bb1f7cfb4ce8942 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:20.496283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.498214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.502875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '111248f8203c4647a65424b43d4d730a', 'request_id': 'req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.512719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '111248f8203c4647a65424b43d4d730a', 'request_id': 'req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.523479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '111248f8203c4647a65424b43d4d730a', 'request_id': 'req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.537776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] The token cache is being invalidate because project 111248f8203c4647a65424b43d4d730a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:20.538068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.538735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c9b26ea-01ee-454e-a57b-5db03b4bba5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.542842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2155] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:20 2022] DELETE /identity/v3/projects/111248f8203c4647a65424b43d4d730a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:20.544290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe0d47-ad89-42b6-a284-0bead94403c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.544469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe0d47-ad89-42b6-a284-0bead94403c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.544775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe0d47-ad89-42b6-a284-0bead94403c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.562801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fdbe0d47-ad89-42b6-a284-0bead94403c7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:20.564020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdbe0d47-ad89-42b6-a284-0bead94403c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.578691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b301d-60b9-4657-b2a5-5c3362593a14 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.584439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.585518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.585948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.586417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] PATH_INFO: `/v3/users/af958351178a4ace9f2da389d90b74e4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.587804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af958351178a4ace9f2da389d90b74e4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.589958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.614690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:20.630689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2156] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:20.632091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.632428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.632649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.651579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af958351178a4ace9f2da389d90b74e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.652762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:20.653958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.660085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af958351178a4ace9f2da389d90b74e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.660910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af958351178a4ace9f2da389d90b74e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.662226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:20.662669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2157] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:20.669516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.670436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.676444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af958351178a4ace9f2da389d90b74e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.677460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6729f15b-50c6-476d-a926-0b0eb30b372b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af958351178a4ace9f2da389d90b74e4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:20.697580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1084/2158] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:20 2022] DELETE /identity/v3/users/af958351178a4ace9f2da389d90b74e4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:20.698917 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.699206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.699484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.719416 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:20.720765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.732992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f24210-ebbf-479e-bbfc-694dcb5ce8b8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.799100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1085/2159] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:20.803756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.814500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550143c3-2c7e-4646-b6c2-48e4432c5baf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.814777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.815409 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.815687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.816012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.818158 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:20.820895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.821871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.822506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.823027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.824056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:20.824515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.826352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.832759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.834214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.843787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1086/2160] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:39:20.844224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2161] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:20.845023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.854069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0aa7a-6d4d-46a6-9b18-38a238a9da4c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.861630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.862290 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.862563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.862837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.863630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.865667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:20.896002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2162] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:20.901282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:20.913117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b886f9c-651a-481d-8c4d-9ccdfae21091 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:20.987177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:20.988339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:20.988827 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:20.989297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:20.990198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:20.993016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.011745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.019039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.037821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec2e64f-5b96-48f0-a7b9-6cf33c7a3a55 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.043980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.046244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.046627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.047017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] PATH_INFO: `/v3/projects/65db607bd69745789d47ef270f233a4e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.048657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65db607bd69745789d47ef270f233a4e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.052439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.057385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1087/2163] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:20 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:21.064393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.075093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8ded0-89e5-4ccd-9c52-768179132bb1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.080372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.081217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.081642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.082140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.083026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.084870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.099948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1088/2164] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:21.104323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.114441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8376482-7c65-440c-a6f0-0ca3840ebab4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.122671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65db607bd69745789d47ef270f233a4e', 'request_id': 'req-eb1911a4-269a-4297-bc61-dc9644953b23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.133138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65db607bd69745789d47ef270f233a4e', 'request_id': 'req-eb1911a4-269a-4297-bc61-dc9644953b23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.141276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65db607bd69745789d47ef270f233a4e', 'request_id': 'req-eb1911a4-269a-4297-bc61-dc9644953b23'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.155942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:21.163717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] The token cache is being invalidate because project 65db607bd69745789d47ef270f233a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:21.163999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.167016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb1911a4-269a-4297-bc61-dc9644953b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.169540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2165] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:21 2022] DELETE /identity/v3/projects/65db607bd69745789d47ef270f233a4e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:21.177952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.178277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.178580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.187540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.188473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.188813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.189013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.189732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.191337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.191719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:21.199249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:21.201310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.202593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1089/2166] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:21.206709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.208520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.211660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2167] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:21.212573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.213962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.226287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c689d-4519-4faf-ada8-2c5fa8189c01 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.226696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582619-c1ec-4a02-bdf6-ac7015884fab admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.231159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.231571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.232096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.232415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.232523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.232687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] PATH_INFO: `/v3/projects/3161f12782024931a2f31ec78e0ddc80/users/93fd81c9079c47a6820899db303a4830/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.232782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.233005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] PATH_INFO: `/v3/projects/7d21540f6ac14ac5a591f0e16a8c6c1c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.234361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d21540f6ac14ac5a591f0e16a8c6c1c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.237764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.255088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3161f12782024931a2f31ec78e0ddc80, user_id=93fd81c9079c47a6820899db303a4830, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.257807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61a27f-0086-4a03-9fbd-0a341c1118d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.289070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2168] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:21 2022] PUT /identity/v3/projects/3161f12782024931a2f31ec78e0ddc80/users/93fd81c9079c47a6820899db303a4830/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:21.290990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.291270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.291545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.310236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d21540f6ac14ac5a591f0e16a8c6c1c', 'request_id': 'req-b1fb5118-9bf5-453d-918b-00795618146f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.317860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d21540f6ac14ac5a591f0e16a8c6c1c', 'request_id': 'req-b1fb5118-9bf5-453d-918b-00795618146f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.329203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:21.330884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.339918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d21540f6ac14ac5a591f0e16a8c6c1c', 'request_id': 'req-b1fb5118-9bf5-453d-918b-00795618146f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.357627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] The token cache is being invalidate because project 7d21540f6ac14ac5a591f0e16a8c6c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:21.357927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.358611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b1fb5118-9bf5-453d-918b-00795618146f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.365859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1090/2169] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:21 2022] DELETE /identity/v3/projects/7d21540f6ac14ac5a591f0e16a8c6c1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:21.367444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.367735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.368016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.404567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2170] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:21.405123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] MFA Rules not processed for user `93fd81c9079c47a6820899db303a4830`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:21.405450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.406769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.407406 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.419211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c4468-b446-479b-aaa5-328154aaa637 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.424021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.424927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.425209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.425481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] PATH_INFO: `/v3/users/a934a9ed1bcc485e80e1f94ae4867480` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.426914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a934a9ed1bcc485e80e1f94ae4867480)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.428891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.474403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1091/2171] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:21.475268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.492745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a934a9ed1bcc485e80e1f94ae4867480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.504965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0b202-37b0-47e0-98a1-8964d3470077 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.509398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a934a9ed1bcc485e80e1f94ae4867480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.510023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a934a9ed1bcc485e80e1f94ae4867480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.510651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a934a9ed1bcc485e80e1f94ae4867480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.511055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.512415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.512415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.512634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.513187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:21.513493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.515617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.524937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a934a9ed1bcc485e80e1f94ae4867480'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:21.532489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1092/2172] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:21.532489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.532489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.538614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2173] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:21 2022] DELETE /identity/v3/users/a934a9ed1bcc485e80e1f94ae4867480 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:21.539417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.540744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.578720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:21.599698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:21.609160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf208d7d-46c4-40f5-b4f0-57190c843855 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.635432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbce0f2-371e-458c-8d6e-49ad5edb8ad2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.642046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.642871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.643734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.644160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.645067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.647407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.693982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2174] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:21.696146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.697090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.697818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.697818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.697991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.699469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.771038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fa09f-756d-436a-857f-66ef68db6cd4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.773849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.775536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.775967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.776665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.776994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.777601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.778010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.779949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.786981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1093/2175] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:21.803726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2176] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:21.808173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.820668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa0162-f3ca-4337-ae3b-919b79971b4c service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.827488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.828390 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.828887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.829233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.830235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.833487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.848729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1094/2177] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:21.852817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.863169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a87a09-570e-4672-a87b-2f9506c8da8e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.868435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.869104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.869380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.869650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.870378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.871993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.883384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2178] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:21.888201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.899755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fa805-f204-48af-bdb3-ee56677a78e3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.904903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:21.905920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.906442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.906442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] PATH_INFO: `/v3/projects/28eaf8647df449ca9c9a1cd8b1cfaa6e/users/3847ea2eff434d1f95a744cda1ec24a9/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.930663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=28eaf8647df449ca9c9a1cd8b1cfaa6e, user_id=3847ea2eff434d1f95a744cda1ec24a9, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:21.937385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd317c61-ba72-4ee2-bc86-97746621fcfb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:21.944097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:21.946729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:21.968253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1095/2179] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:21 2022] PUT /identity/v3/projects/28eaf8647df449ca9c9a1cd8b1cfaa6e/users/3847ea2eff434d1f95a744cda1ec24a9/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:21.968470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdd120-d986-4766-ad47-85fb015af3b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:21.970113 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:21.970396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:21.970666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:21.991217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:21.992573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.041597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.042509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.042790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.043067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] PATH_INFO: `/v3/projects/ff9c21c940b943458ee6feafbfbcb273` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.044287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9c21c940b943458ee6feafbfbcb273)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.046217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.063338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1096/2180] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:22.071203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.072979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.086799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:22.107958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9c21c940b943458ee6feafbfbcb273', 'request_id': 'req-769a4f25-1a07-4375-a295-442f3bcd32eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:22.114103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:22.117875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9c21c940b943458ee6feafbfbcb273', 'request_id': 'req-769a4f25-1a07-4375-a295-442f3bcd32eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:22.131199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9c21c940b943458ee6feafbfbcb273', 'request_id': 'req-769a4f25-1a07-4375-a295-442f3bcd32eb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:22.146864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] The token cache is being invalidate because project ff9c21c940b943458ee6feafbfbcb273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:22.147241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:22.150054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-769a4f25-1a07-4375-a295-442f3bcd32eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:22.150407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2181] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:21 2022] DELETE /identity/v3/projects/ff9c21c940b943458ee6feafbfbcb273 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:22.163442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1342f5-21bf-46ea-9441-7df15bb46bbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.163442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1342f5-21bf-46ea-9441-7df15bb46bbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.163442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1342f5-21bf-46ea-9441-7df15bb46bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.163764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0dd6a-a9c2-43c5-951d-927814c2ed4b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.179598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2b1342f5-21bf-46ea-9441-7df15bb46bbb None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:22.180937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b1342f5-21bf-46ea-9441-7df15bb46bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.228414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.230716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.230716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.230716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.233818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.276643 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2182] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:22.285699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.287326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.287630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.306869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:22.307124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.308092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.308630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.318470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1097/2183] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:22.319621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2184] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:22.320605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.330417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17ca-0195-4030-870d-33e493ac92dd service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.335447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.336138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.336465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.336677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.337449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:22.337931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.339612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.353715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1098/2185] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:39:22.358048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.367389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e627f03-69c5-4d1e-944c-fbee923d20bc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.372142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.372810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.373088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.373568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.374037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.375944 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.407612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2186] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:22.409089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.420770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70643f21-157d-4962-8669-2c5d45f70e89 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.483565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.484220 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.485140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.485490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.486368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.488888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.498889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:22.521284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1099/2187] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/users => generated 297 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:22.526381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.538351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9a9b6-362e-45e1-aa0e-7e686812c298 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.543019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.543871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.544182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.544462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.545737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.546913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.559760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2188] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:22.570088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.596564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bc668-ce75-4029-bdfd-5e3dc0c6e712 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.604919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.605749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.606132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.606508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.607344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.610029 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbe63f-ecd1-4784-9d61-c067137259e9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.622235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1100/2189] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:22.627223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.638897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683e16b-205d-499d-a1f4-5f72812d3183 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.646734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.647851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.648354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.648843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] PATH_INFO: `/v3/projects/250d78ed111c40078ccaa096a8092c41/users/f486f49d14f74e33baeb277ee1f910d5/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.671285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250d78ed111c40078ccaa096a8092c41, user_id=f486f49d14f74e33baeb277ee1f910d5, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.674330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.706394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2190] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:22 2022] PUT /identity/v3/projects/250d78ed111c40078ccaa096a8092c41/users/f486f49d14f74e33baeb277ee1f910d5/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:22.719271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.719713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.720003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.758576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] MFA Rules not processed for user `f486f49d14f74e33baeb277ee1f910d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:22.760272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:22.866428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1101/2191] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:22.867874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.943195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120a84f-56f4-4d23-9574-ff196439a652 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:22.948135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:22.948810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:22.949106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:22.949367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:22.950066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:22.952060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d59a2-40a4-48e9-ab52-96c893bae54a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:22.982330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2192] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:22.986012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:22.998349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c06d19-f990-4013-8eb0-473928f58ead None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.083910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.084227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.084526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.085105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.085793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.086068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.086618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.087571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.089636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.093303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:23.124926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1102/2193] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:22 2022] POST /identity/v3/users => generated 303 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:23.126282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] MFA Rules not processed for user `3847ea2eff434d1f95a744cda1ec24a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:23.127495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:23.128761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.140062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69e80-806a-4e90-b15a-890560a9184a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.145294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.146009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.146287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.146558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.147282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.148931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.160059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1103/2194] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:23.164022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.177790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1a38d-889c-43a5-a8a4-06f88118a71b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.186689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.188683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.189910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.190355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] PATH_INFO: `/v3/projects/1f6171ff8bf844d9833dc1eedebce450/users/5f14bd5767fa4e638260bd6e04bf30b8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.211532 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6171ff8bf844d9833dc1eedebce450, user_id=5f14bd5767fa4e638260bd6e04bf30b8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.214446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.232306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1104/2195] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:23 2022] PUT /identity/v3/projects/1f6171ff8bf844d9833dc1eedebce450/users/5f14bd5767fa4e638260bd6e04bf30b8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:23.235346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2196] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:23.236554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.240346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.242137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:23.259667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d78407-78f5-4f88-a038-3e51faca9c21 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.279499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:23.329585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d856be0-ef10-4aa4-aefb-394e74e64a1b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.347414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.348234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.348498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.348780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.350678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.350961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.350961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.350961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.351187 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.351268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.351909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.353918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.362804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1105/2197] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:23.367595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.383384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8703fa-c2ab-4ab8-9d05-e132e5a7bf4e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.389139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.390292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.390573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.390848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] PATH_INFO: `/v3/projects/1f6171ff8bf844d9833dc1eedebce450/users/5f14bd5767fa4e638260bd6e04bf30b8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.392951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6171ff8bf844d9833dc1eedebce450, user_id=5f14bd5767fa4e638260bd6e04bf30b8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.396609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b6a9b-59cb-4604-987e-ec251d05bba4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.410639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2198] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:23 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:23.412826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1106/2199] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:23 2022] PUT /identity/v3/projects/1f6171ff8bf844d9833dc1eedebce450/users/5f14bd5767fa4e638260bd6e04bf30b8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:23.416212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.423734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.424054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.424328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.478322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] MFA Rules not processed for user `5f14bd5767fa4e638260bd6e04bf30b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:23.479610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:23.536953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd4a1-135e-48fa-950b-589ffec69fc6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.541746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.542457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.542742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.543015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.543709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.545732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.551435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1107/2200] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:23.562860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.564373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:23.567493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2201] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:23 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:23.571037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.581215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5f873-bd75-4e8c-b6ee-00e42ac2d385 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.586399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.587086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.587392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.587665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.588419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.590072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.592993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:23.604241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2202] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:23.608598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.619415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb0839-c963-4b4f-a256-42c8caa33090 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.624591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.625488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.625760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.626051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] PATH_INFO: `/v3/projects/f9bde3047af445608f3436d09ae05561/users/8ad193b8e98948e58932bc4e39e5e9ca/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.633184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb0f88-de97-4a10-a3c2-00e9f7d93389 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.637371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.638014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.638289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.638557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.643402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bde3047af445608f3436d09ae05561, user_id=8ad193b8e98948e58932bc4e39e5e9ca, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.646647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.649082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.650962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.664217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2203] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:23 2022] PUT /identity/v3/projects/f9bde3047af445608f3436d09ae05561/users/8ad193b8e98948e58932bc4e39e5e9ca/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:23.664405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1108/2204] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:23.669027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.711355 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.724399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93def413-ec86-46a2-a9c4-01a7b379d978 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.753189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292001f9-feda-439b-99d4-98a8c7b43d5f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.758092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.758775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.759051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.759325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.760080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.762422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.773438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:23.794686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2205] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:23.797361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.798326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.798889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.799004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:23.799454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.829645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ec33-bdf1-44dd-9134-c8764b1e7e45 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:23.836650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:23.837624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:23.837906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:23.838182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] PATH_INFO: `/v3/projects/f9bde3047af445608f3436d09ae05561/users/8ad193b8e98948e58932bc4e39e5e9ca/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:23.840381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bde3047af445608f3436d09ae05561, user_id=8ad193b8e98948e58932bc4e39e5e9ca, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.843885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.862289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2206] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:23 2022] PUT /identity/v3/projects/f9bde3047af445608f3436d09ae05561/users/8ad193b8e98948e58932bc4e39e5e9ca/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:23.905859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:23.907653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e266c-574f-45fd-8d7a-401cbdbcda24 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:23.919969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1109/2207] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:24.572937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:24.574741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:24.674106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88b3ab-89d1-40df-9ee3-fe4fdad68b37 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:24.679260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:24.680336 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:24.680674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:24.681122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] PATH_INFO: `/v3/users/93fd81c9079c47a6820899db303a4830` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:24.682341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93fd81c9079c47a6820899db303a4830)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:24.684715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:24.747837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93fd81c9079c47a6820899db303a4830'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:24.757830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93fd81c9079c47a6820899db303a4830'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:24.758789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93fd81c9079c47a6820899db303a4830'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:24.759761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93fd81c9079c47a6820899db303a4830'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:24.767478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:24.781261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93fd81c9079c47a6820899db303a4830'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:24.804149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2208] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:24 2022] DELETE /identity/v3/users/93fd81c9079c47a6820899db303a4830 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:24.807889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a11ef-f078-4f7c-a2e3-82a0245fd911 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:24.809252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a11ef-f078-4f7c-a2e3-82a0245fd911 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:24.809757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a11ef-f078-4f7c-a2e3-82a0245fd911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:24.832469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7f2a11ef-f078-4f7c-a2e3-82a0245fd911 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:24.833846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f2a11ef-f078-4f7c-a2e3-82a0245fd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:24.899643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1110/2209] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:24.910018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:24.911142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:24.974379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47773591-7097-447d-b9b1-0bcce0b490a2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:24.986207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:24.986905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:24.987177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:24.987282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:24.987453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:24.987549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:24.987796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:24.997472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:24.998929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:25.009399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2210] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:25.027607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] MFA Rules not processed for user `8ad193b8e98948e58932bc4e39e5e9ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:25.031139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.126012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1111/2211] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:25.139638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:25.141493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.240607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee393bf-4664-4baf-979e-81753d3e95c9 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:25.244257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:25.244915 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:25.245191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:25.245463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:25.247280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.310605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:25.312101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7921ae6-0459-4ee6-a547-a69e22c292fb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:25.338510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2212] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:25.627957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:25.629526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.643648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001aa2-de1e-45be-8573-048f0012fb87 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:25.650519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:25.652179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:25.652841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:25.653367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] PATH_INFO: `/v3/projects/3161f12782024931a2f31ec78e0ddc80` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:25.661461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3161f12782024931a2f31ec78e0ddc80)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:25.668834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:25.697831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:25.781838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3161f12782024931a2f31ec78e0ddc80', 'request_id': 'req-b47daaf8-3a01-44af-9b16-5a1273618f7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:25.803475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3161f12782024931a2f31ec78e0ddc80', 'request_id': 'req-b47daaf8-3a01-44af-9b16-5a1273618f7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:25.822494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3161f12782024931a2f31ec78e0ddc80', 'request_id': 'req-b47daaf8-3a01-44af-9b16-5a1273618f7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:25.838483 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] The token cache is being invalidate because project 3161f12782024931a2f31ec78e0ddc80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:25.839436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:25.845435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b47daaf8-3a01-44af-9b16-5a1273618f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:25.850584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1112/2213] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:25 2022] DELETE /identity/v3/projects/3161f12782024931a2f31ec78e0ddc80 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:25.855651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:25.856172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:25.856252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:25.880537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:25.880537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.916469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:25.954070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2214] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:25.962080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:25.962370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:25.962646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:25.986702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:25.988093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:25.996860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1113/2215] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:26.001443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.012061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b750c9c-01bc-4a11-bc07-2eebd4f7f487 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.016740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.017404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.017680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.017950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.018678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:26.018980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.020801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.039323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2216] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:39:26.040600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.050382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b34a6-b30f-4a5e-8fd1-f32f307b040a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.055362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.056165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.056505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.056831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.057614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.059605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.087886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1114/2217] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:26 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:26.092318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.102354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b6a3b-84bb-4a9b-9f3d-7684d32b4dc1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.159969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.160703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.160997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.161269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.161952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.163874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.184838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2218] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:26 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:26.189631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.201552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678120f1-2a28-4644-8367-bc99f757dbc9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.206717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.207663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.207977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.208339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.208997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.210756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.226304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1115/2219] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:26.231867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.246015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f31a3-dbd5-4177-9df6-41abcf8a1a39 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.251083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.252114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.252493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.252880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.253974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.255832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5bbbd7-36d7-4eb9-8015-f18b2c3385c2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.268006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2220] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:26.271784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.296602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab36981-abf9-43fa-9a18-336fded6ca1f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.302608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.303790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.304095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.304399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] PATH_INFO: `/v3/projects/a566b21e5554499ca67a34aed183bf09/users/16fcb4a9fcbf4c3daca7ac1d3df7c8f3/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.321081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a566b21e5554499ca67a34aed183bf09, user_id=16fcb4a9fcbf4c3daca7ac1d3df7c8f3, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.323871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.356477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1116/2221] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:26 2022] PUT /identity/v3/projects/a566b21e5554499ca67a34aed183bf09/users/16fcb4a9fcbf4c3daca7ac1d3df7c8f3/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:26.360740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.361351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.362453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.392410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] MFA Rules not processed for user `16fcb4a9fcbf4c3daca7ac1d3df7c8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:26.394501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:26.475543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2222] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:26.487782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.489935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:26.569618 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188117ba-6129-40ee-b398-0c8e858a9cb0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.576880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.577800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.578569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.579046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.590055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:26.591738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:26.601819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1117/2223] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:26 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:26.881389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:26.892635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9ec16-7d92-40c1-83d5-3174e88fb438 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:26.897658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:26.898833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:26.898833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:26.898833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:26.901264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:26.951891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:27.000991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:27.003923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62be3-9cb7-46c0-b56d-60eb624b83c3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:27.024837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2224] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:27.521613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:27.523325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:27.576066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:27.605360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12244c5-4be3-4805-ad96-ea6eb0c2c42e service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:27.610719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:27.611781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:27.612014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:27.612314 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] PATH_INFO: `/v3/users/f486f49d14f74e33baeb277ee1f910d5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:27.613761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f486f49d14f74e33baeb277ee1f910d5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:27.616371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:27.693915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f486f49d14f74e33baeb277ee1f910d5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:27.703162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f486f49d14f74e33baeb277ee1f910d5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:27.704349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f486f49d14f74e33baeb277ee1f910d5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:27.715388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f486f49d14f74e33baeb277ee1f910d5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:27.716472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f486f49d14f74e33baeb277ee1f910d5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:27.742878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1118/2225] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:27 2022] DELETE /identity/v3/users/f486f49d14f74e33baeb277ee1f910d5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:27.747472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21e55-9438-4924-9e3e-84bbdc608ed9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:27.747885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21e55-9438-4924-9e3e-84bbdc608ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:27.748339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b21e55-9438-4924-9e3e-84bbdc608ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:27.772888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-07b21e55-9438-4924-9e3e-84bbdc608ed9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:27.774088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07b21e55-9438-4924-9e3e-84bbdc608ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:27.855787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2226] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:27.867072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:27.868952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:27.930850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c505-2907-452e-9696-9629130f7c55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:27.935933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:27.936620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:27.936898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:27.937169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:27.947442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:27.948951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:27.958970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1119/2227] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:28.174591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:28.175511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:28.175511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:28.198431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] MFA Rules not processed for user `0f0285314f124d228986e7f638833d04`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:28.199708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:28.265674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2228] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:28.352306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:28.353824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:28.368568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e940354-dcad-4ef6-bc14-b3fbe167e14c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:28.373473 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:28.374707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:28.375264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:28.375678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] PATH_INFO: `/v3/projects/250d78ed111c40078ccaa096a8092c41` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:28.377074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=250d78ed111c40078ccaa096a8092c41)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:28.379176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:28.467300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250d78ed111c40078ccaa096a8092c41', 'request_id': 'req-6ea0b47f-640d-43ba-be05-20e085054f4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.472359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:28.476196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250d78ed111c40078ccaa096a8092c41', 'request_id': 'req-6ea0b47f-640d-43ba-be05-20e085054f4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.489296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250d78ed111c40078ccaa096a8092c41', 'request_id': 'req-6ea0b47f-640d-43ba-be05-20e085054f4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.506543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] The token cache is being invalidate because project 250d78ed111c40078ccaa096a8092c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:28.506889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.507482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.512082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1120/2229] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:28 2022] DELETE /identity/v3/projects/250d78ed111c40078ccaa096a8092c41 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:28.513198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:28.526847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac7917-ceda-43c5-9fa9-822dd1567219 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:28.659068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:28.660633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:28.661238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:28.661843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] PATH_INFO: `/v3/users/5f14bd5767fa4e638260bd6e04bf30b8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:28.663446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f14bd5767fa4e638260bd6e04bf30b8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:28.665674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:28.679646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:28.741271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f14bd5767fa4e638260bd6e04bf30b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.762208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f14bd5767fa4e638260bd6e04bf30b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.764477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f14bd5767fa4e638260bd6e04bf30b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.766771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f14bd5767fa4e638260bd6e04bf30b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.786333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07093232-e27b-4ba9-a082-19fad34f33c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f14bd5767fa4e638260bd6e04bf30b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:28.805815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2230] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:28 2022] DELETE /identity/v3/users/5f14bd5767fa4e638260bd6e04bf30b8 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:29.176982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.178478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.192005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0b47f-640d-43ba-be05-20e085054f4b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.263313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.264546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.265007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.265453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] PATH_INFO: `/v3/projects/1f6171ff8bf844d9833dc1eedebce450` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.267438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6171ff8bf844d9833dc1eedebce450)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.270339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.328274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6171ff8bf844d9833dc1eedebce450', 'request_id': 'req-a999317e-9111-402e-9307-51c639a4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:29.336350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6171ff8bf844d9833dc1eedebce450', 'request_id': 'req-a999317e-9111-402e-9307-51c639a4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:29.346072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6171ff8bf844d9833dc1eedebce450', 'request_id': 'req-a999317e-9111-402e-9307-51c639a4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:29.359148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] The token cache is being invalidate because project 1f6171ff8bf844d9833dc1eedebce450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:29.359439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:29.360139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a999317e-9111-402e-9307-51c639a4235c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:29.361787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1121/2231] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:29 2022] DELETE /identity/v3/projects/1f6171ff8bf844d9833dc1eedebce450 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:29.367304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.368482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.368872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.391822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:29.393360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.479585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2232] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:29.488498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.488858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.489216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.508587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:29.509887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.517913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1122/2233] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:29.522176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.532690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e9fe5a-2fb1-4be6-9a60-58e2b21feb51 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.537545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.538214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.538490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.538761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.539491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:29.539798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.541472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.553504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2234] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:29.557760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.566938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19ec60-bbae-4c54-be66-dc6d7ada4815 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.571525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.572224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.572500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.572768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.573465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.575373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.602409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1123/2235] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:29.606546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.617259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97551692-56bf-415c-b99e-a6eff2e55ee9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.684605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.685295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.685577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.685853 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.686546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.688651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.713258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2236] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:29.714474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.725080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e42180-6352-40b9-b6a6-8aec5c0394a6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.730116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.730781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.731061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.731331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.732088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.733700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72ded-1bfe-408e-b809-7533571ba4cb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.744958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1124/2237] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:29.749121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.757036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:29.761762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f34cfaa-49a3-4e73-b9e8-cbcafad36a7d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.763470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35492e0-8d33-4066-bc9d-5fc4842ce809 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.763988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35492e0-8d33-4066-bc9d-5fc4842ce809 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.764616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35492e0-8d33-4066-bc9d-5fc4842ce809 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.766209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1125/2238] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:39:29.769374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.770183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.770477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.771367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.772157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.773919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.777184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd571f-ff17-4ac2-bec6-f2700d815389 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.777533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd571f-ff17-4ac2-bec6-f2700d815389 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.777881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd571f-ff17-4ac2-bec6-f2700d815389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.791110 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2239] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:29.792840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.798724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-77bd571f-ff17-4ac2-bec6-f2700d815389 None None] MFA Rules not processed for user `3e5a44c30a0b430f90cabf9b3084ec28`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:29.799989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-77bd571f-ff17-4ac2-bec6-f2700d815389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.805487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896ad11-76fc-42be-bc18-10d67455dcfc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.810727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.812182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.812182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.812592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] PATH_INFO: `/v3/projects/64e5d5f9b0014e6d8829737e139a5d1d/users/0436a3494d084bc599ad088ce477bd60/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.830222 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e5d5f9b0014e6d8829737e139a5d1d, user_id=0436a3494d084bc599ad088ce477bd60, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.833235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:29.835096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:29.860970 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2240] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:29 2022] PUT /identity/v3/projects/64e5d5f9b0014e6d8829737e139a5d1d/users/0436a3494d084bc599ad088ce477bd60/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:29.861858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:29.864000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.892293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1126/2241] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:29.893649 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.893935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.894208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.923295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] MFA Rules not processed for user `0436a3494d084bc599ad088ce477bd60`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:29.924607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:29.941326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e75464-a297-4cfb-ac62-a15e852fcad6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:29.946580 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:29.947503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:29.947803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:29.948168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] PATH_INFO: `/v3/users/e9f6a35cac50431d9468b51c9259bdb5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:29.949402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f6a35cac50431d9468b51c9259bdb5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:29.951353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.005211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1127/2242] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:30.005211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.006424 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.006638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f6a35cac50431d9468b51c9259bdb5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.015364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f6a35cac50431d9468b51c9259bdb5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.015978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f6a35cac50431d9468b51c9259bdb5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.016613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f6a35cac50431d9468b51c9259bdb5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.029476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f6a35cac50431d9468b51c9259bdb5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.042124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2243] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:29 2022] DELETE /identity/v3/users/e9f6a35cac50431d9468b51c9259bdb5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:30.043219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.044639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.087519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3b513-140b-4bc7-b91b-4ef587e3ba0a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.105213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:30.109502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35660858-1b51-486c-883a-688c42a204b5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.114357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.115032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.115465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.116261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.119189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.155041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:30.163574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.164557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.164915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.165195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.167243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.179947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.181859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.195760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1128/2244] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:30.203199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.204999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a69b7c-bdd3-4790-9d2f-93bec896d0b4 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.218603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2245] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:30.251922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.253571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.329752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc873f7-7252-4a0e-835c-6e0eaad3906b service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.334891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.335761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.336063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.336338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] PATH_INFO: `/v3/projects/f86f8a870f9146489a3ef1716094e066` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.337563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86f8a870f9146489a3ef1716094e066)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.339593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.393059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86f8a870f9146489a3ef1716094e066', 'request_id': 'req-8ee0a114-213d-4966-b629-b8961dbfb6ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.399349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86f8a870f9146489a3ef1716094e066', 'request_id': 'req-8ee0a114-213d-4966-b629-b8961dbfb6ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.409038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86f8a870f9146489a3ef1716094e066', 'request_id': 'req-8ee0a114-213d-4966-b629-b8961dbfb6ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.423014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] The token cache is being invalidate because project f86f8a870f9146489a3ef1716094e066 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:30.423301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.424004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ee0a114-213d-4966-b629-b8961dbfb6ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:30.425699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1129/2246] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:30 2022] DELETE /identity/v3/projects/f86f8a870f9146489a3ef1716094e066 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:30.432821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.433307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.433807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.457074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:30.458274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.571165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2247] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:30.575862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.576199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.577160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.597171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:30.598426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:30.606230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1130/2248] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:30.610436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.620363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59be037-4bba-4323-88ea-77635b21e4b7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.625945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.626602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.626879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.627197 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.627860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:30.628194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.629829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.641946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2249] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:30.646526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.656950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59345d0-652b-4737-9144-ce2856f11ef2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.662027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.662721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.663000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.663311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.664013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.665957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.693391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1131/2250] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:30 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:30.697693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.707843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be010e-9705-4f3d-87cd-54813d807c31 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.765094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.765790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.766074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.766346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.767030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.769088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.790740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2251] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:30 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:30.795091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.805753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db634303-db44-4d40-9a03-154b8de5a561 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.812907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.813875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.814240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.814592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.815576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.817388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.828307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1132/2252] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:30.832231 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.842338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdf2974-7c2f-466b-a628-e4573efbc74e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.847097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.847851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.848159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.848432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.849162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.850776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fffd46-1211-4abb-96c2-72d625b1d5b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.860598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2253] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:30.864996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:30.875025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e59abc-311e-4970-b6f9-0925391fea38 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:30.880213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:30.881091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.881371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.881640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] PATH_INFO: `/v3/projects/42166c9c0aae47cd8ed3570e71c4602a/users/813b39c57d6f472e95596c21e972c20e/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.896926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42166c9c0aae47cd8ed3570e71c4602a, user_id=813b39c57d6f472e95596c21e972c20e, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:30.899597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:30.914872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1133/2254] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:30 2022] PUT /identity/v3/projects/42166c9c0aae47cd8ed3570e71c4602a/users/813b39c57d6f472e95596c21e972c20e/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:30.926040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:30.926403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:30.926785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:30.946096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:30.956439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] MFA Rules not processed for user `813b39c57d6f472e95596c21e972c20e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:30.957775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:31.029006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2255] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:31.040230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:31.042354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:31.063843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:31.119276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc21d77-083c-453a-ac36-35dbb26323eb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:31.125159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:31.125802 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:31.126077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:31.126346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:31.134230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:31.135841 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb9949-4487-4173-b33f-67ab9ac1a474 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:31.144744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1134/2256] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:31.349519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:31.413461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89042c8-dfa7-4879-9788-c7f1b6c13d1e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:31.418849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:31.419722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:31.420001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:31.420380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] PATH_INFO: `/v3/users/813b39c57d6f472e95596c21e972c20e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:31.421609 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813b39c57d6f472e95596c21e972c20e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:31.423563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:31.476736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813b39c57d6f472e95596c21e972c20e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:31.484233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813b39c57d6f472e95596c21e972c20e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:31.484940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813b39c57d6f472e95596c21e972c20e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:31.485598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813b39c57d6f472e95596c21e972c20e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:31.498240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813b39c57d6f472e95596c21e972c20e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:31.510957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2257] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:31 2022] DELETE /identity/v3/users/813b39c57d6f472e95596c21e972c20e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:31.515615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:31.516128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:31.516255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:31.534423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:31.535678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:31.598760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1135/2258] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:31.616485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:31.617946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:31.638691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:31.686482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcf7e5-b159-44c4-9dec-bce74b6b8635 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:31.691151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:31.692801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:31.693091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:31.693364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:31.703452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:31.704975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0187c27-7510-4cca-be29-9fc1b525a71c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:31.725424 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2259] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:32.043360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.044944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:32.063025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583138a2-7cb1-4b4f-b5d3-0e96618bcdd3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.070460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.071601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.071887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.072230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] PATH_INFO: `/v3/projects/42166c9c0aae47cd8ed3570e71c4602a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.074228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] RBAC: Authorizing `identity:delete_project(project_id=42166c9c0aae47cd8ed3570e71c4602a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.076653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.127414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42166c9c0aae47cd8ed3570e71c4602a', 'request_id': 'req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:32.132695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42166c9c0aae47cd8ed3570e71c4602a', 'request_id': 'req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:32.140735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42166c9c0aae47cd8ed3570e71c4602a', 'request_id': 'req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:32.153977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] The token cache is being invalidate because project 42166c9c0aae47cd8ed3570e71c4602a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:32.154296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:32.154987 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ad0eb20-d8ef-4e65-a36f-28f05a0e0eba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:32.156625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1136/2260] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:32 2022] DELETE /identity/v3/projects/42166c9c0aae47cd8ed3570e71c4602a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:32.162489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.162793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.163152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.189337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:32.190555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:32.252177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2261] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:32.260259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.260548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.260828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.280350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:32.281527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:32.290911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1137/2262] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:32.295173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.305224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c0be6-75fe-4e07-99fa-40b34867304e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.309935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.310593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.310908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.311159 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.311901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:32.312247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.313948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.326783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2263] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:32.331258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.341382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2157038-1a28-4320-a349-b4499af89f56 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.353343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.354156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.354331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.356047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.356794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.358735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.381757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:32.390287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1138/2264] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:32.394326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.405637 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380ce43-8a1d-4c85-bac5-7df4ec754b3c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.470865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.471558 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.471835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.472142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.472958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.474937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.497351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2265] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:32.501461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.511968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893435b-cfa2-4ebe-b35c-b179c773b5d2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.516832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.517538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.517833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.518103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.518823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.520504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.531370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1139/2266] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:32.537034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.550696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1f0db-87ac-4f1f-bad5-b2fec54dfc3e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.556252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.557212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.557483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.557755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.558561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.560383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf38b5-345e-45b9-b935-8cd3dba588f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.571555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2267] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:32.575628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.585829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52327078-debb-49b0-b115-2dbc6150dbf3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.591152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.592052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.592357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.592634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] PATH_INFO: `/v3/projects/7e7f9411b9424065b75fbad0d913ec46/users/aeddd5624c0344159614c70bbe34cb3c/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.607211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7f9411b9424065b75fbad0d913ec46, user_id=aeddd5624c0344159614c70bbe34cb3c, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.609934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.626942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1140/2268] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:32 2022] PUT /identity/v3/projects/7e7f9411b9424065b75fbad0d913ec46/users/aeddd5624c0344159614c70bbe34cb3c/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:32.639657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.640117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.640390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.671616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] MFA Rules not processed for user `aeddd5624c0344159614c70bbe34cb3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:32.672835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:32.690270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:32.737864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2269] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:32.742551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.825038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b27c7-bc7b-41e7-adcb-4c5da27b42a6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.841168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.841168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.841168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.841168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.841508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.842196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:32.895448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1141/2270] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/projects => generated 363 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:32.899545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:32.913426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23032f8b-dbe5-4326-90db-c0f6ee677c68 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:32.977431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:32.978134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:32.978496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:32.978775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:32.979464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:32.981479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.003888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2271] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:32 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:33.007978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.018670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5bdbc1-0179-424c-a6bb-abb0ddec9349 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.023829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.024684 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.025084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.025751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.026620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.028261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.047333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1142/2272] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:33.047491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.056246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8a2e8-5648-4958-a91a-7562c49605d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.062337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.063274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.063571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.063825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] PATH_INFO: `/v3/projects/92c74501c1254053ba7b9256a545f64b/users/e6c0d6fc498c438e8d0cf735d26f77c9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.080800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c74501c1254053ba7b9256a545f64b, user_id=e6c0d6fc498c438e8d0cf735d26f77c9, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.083774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.109933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2273] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:33 2022] PUT /identity/v3/projects/92c74501c1254053ba7b9256a545f64b/users/e6c0d6fc498c438e8d0cf735d26f77c9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:33.114934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.130244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe8340-a906-487a-8771-71f03bd15187 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.208980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.209760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.210079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.211096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.211912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.213774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e32fa-a9c1-4da0-b6ed-6dfb69600d31 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.224392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1143/2274] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:33.228952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.240107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cf3f1-e93d-4f98-80a1-1a1c658e3686 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.245766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.246734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.247010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.247284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] PATH_INFO: `/v3/projects/92c74501c1254053ba7b9256a545f64b/users/e6c0d6fc498c438e8d0cf735d26f77c9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.249317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c74501c1254053ba7b9256a545f64b, user_id=e6c0d6fc498c438e8d0cf735d26f77c9, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.251954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.264343 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2275] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:33 2022] PUT /identity/v3/projects/92c74501c1254053ba7b9256a545f64b/users/e6c0d6fc498c438e8d0cf735d26f77c9/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:33.275990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.276345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.276619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.303348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] MFA Rules not processed for user `e6c0d6fc498c438e8d0cf735d26f77c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:33.304600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:33.324200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:33.374657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1144/2276] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:33.386078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.388145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:33.448180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d389cea-050c-4312-a428-4feeede131e7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.452019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.452587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.452865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.453136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.462455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.463916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9954e1ab-7451-418b-a305-f4afde3721d5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:33.473513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2277] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:33.902216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:33.967218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6ea23-3802-41c8-acb5-1c08c3d61948 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:33.971921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:33.973427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:33.973709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:33.973986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] PATH_INFO: `/v3/users/aeddd5624c0344159614c70bbe34cb3c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:33.975254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeddd5624c0344159614c70bbe34cb3c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:33.977260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:34.026896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeddd5624c0344159614c70bbe34cb3c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.034960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeddd5624c0344159614c70bbe34cb3c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.036268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeddd5624c0344159614c70bbe34cb3c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.043694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeddd5624c0344159614c70bbe34cb3c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.044834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeddd5624c0344159614c70bbe34cb3c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.064184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1145/2278] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:33 2022] DELETE /identity/v3/users/aeddd5624c0344159614c70bbe34cb3c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:34.069005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:34.069505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:34.069607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:34.087323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:34.088561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:34.093616 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:34.153166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2279] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:34.164101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:34.165616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:34.211921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:34.230339 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f719c-6122-45f2-9a26-8a9256306ab1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:34.233972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:34.234667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:34.235123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:34.235315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:34.244609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:34.246042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:34.255493 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1146/2280] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:34.536991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:34.539847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:34.554246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76899a-8fab-43bf-ae0d-46ae0eacfdea None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:34.561771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:34.563183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:34.563772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:34.564253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] PATH_INFO: `/v3/projects/7e7f9411b9424065b75fbad0d913ec46` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:34.566896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7f9411b9424065b75fbad0d913ec46)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:34.570828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:34.636575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7f9411b9424065b75fbad0d913ec46', 'request_id': 'req-8e6568f9-aa22-4dca-8560-cc0e14d917cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.644938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7f9411b9424065b75fbad0d913ec46', 'request_id': 'req-8e6568f9-aa22-4dca-8560-cc0e14d917cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.659754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7f9411b9424065b75fbad0d913ec46', 'request_id': 'req-8e6568f9-aa22-4dca-8560-cc0e14d917cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.672740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] The token cache is being invalidate because project 7e7f9411b9424065b75fbad0d913ec46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:34.673027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.673673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.675269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2281] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:34 2022] DELETE /identity/v3/projects/7e7f9411b9424065b75fbad0d913ec46 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:34.679212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:34.690959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c13737-27fd-4587-ac32-32874bb6ca54 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:34.754137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:34.755023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:34.755295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:34.755560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] PATH_INFO: `/v3/users/e6c0d6fc498c438e8d0cf735d26f77c9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:34.756759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c0d6fc498c438e8d0cf735d26f77c9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:34.758672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:34.806548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c0d6fc498c438e8d0cf735d26f77c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.814229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c0d6fc498c438e8d0cf735d26f77c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.814853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c0d6fc498c438e8d0cf735d26f77c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.821885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c0d6fc498c438e8d0cf735d26f77c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.822503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b6af6d4-db70-498a-9ec8-7101b8b04026 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c0d6fc498c438e8d0cf735d26f77c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:34.839456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1147/2282] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:34 2022] DELETE /identity/v3/users/e6c0d6fc498c438e8d0cf735d26f77c9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:35.122567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.124218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:35.137943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6568f9-aa22-4dca-8560-cc0e14d917cd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.195116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.195985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.196356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.196565 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] PATH_INFO: `/v3/projects/92c74501c1254053ba7b9256a545f64b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.197814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c74501c1254053ba7b9256a545f64b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.199756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.242215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:35.253127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c74501c1254053ba7b9256a545f64b', 'request_id': 'req-f5d727e0-633c-4a63-aabb-c8e72f83c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:35.258840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c74501c1254053ba7b9256a545f64b', 'request_id': 'req-f5d727e0-633c-4a63-aabb-c8e72f83c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:35.266621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c74501c1254053ba7b9256a545f64b', 'request_id': 'req-f5d727e0-633c-4a63-aabb-c8e72f83c36c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:35.279228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] The token cache is being invalidate because project 92c74501c1254053ba7b9256a545f64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:35.279508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:35.280165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f5d727e0-633c-4a63-aabb-c8e72f83c36c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:35.281770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2283] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:35 2022] DELETE /identity/v3/projects/92c74501c1254053ba7b9256a545f64b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:35.286967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.287470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.287558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.306311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:35.307506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:35.364481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:35.374983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1148/2284] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:35.382256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.382631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.383023 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.403950 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:35.405172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:35.414752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2285] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:35.426584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.436897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ebfc0-be8d-41af-b6d6-8fc98d688074 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.442235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.442891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.443173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.443449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.444231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:35.444540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.446208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.458190 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1149/2286] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:35.462495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.472308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000806-bfa5-492e-87e0-b36b2c53d302 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.477831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.478484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.478756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.479038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.479706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.481650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.514772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2287] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:35.515886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.526494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f72488a-31b7-4fef-8853-e84d9b5222a5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.586446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.587097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.587376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.587647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.588384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.590313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.611541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1150/2288] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:35.615871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.625989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b207aa-6622-4114-90d9-692c0034d14f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.630897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.631555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.631833 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.632135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.632891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.634497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.644994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2289] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:35.648868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.660279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4438ad36-f9a3-437d-a804-4bf6f14158e3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.665392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.666162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.666447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.666721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.667450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.669216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da11b40-c15b-4d9f-b6b9-d482c1facb8b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.680346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1151/2290] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:35.684988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.695664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033e295-0b61-46e4-b599-ab53c9496c4c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.700681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.701577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.701888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.702189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] PATH_INFO: `/v3/projects/5c6aa6527ffc4cfcadbc899f694919a3/users/3641b99149e9486fa3777eb0b45c6085/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.717321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6aa6527ffc4cfcadbc899f694919a3, user_id=3641b99149e9486fa3777eb0b45c6085, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.720105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.741787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2291] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:35 2022] PUT /identity/v3/projects/5c6aa6527ffc4cfcadbc899f694919a3/users/3641b99149e9486fa3777eb0b45c6085/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:35.745828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.746147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.746561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.783643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] MFA Rules not processed for user `3641b99149e9486fa3777eb0b45c6085`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:35.784888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:35.848065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1152/2292] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:35.852967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.917397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6050b2c2-46ea-4ae6-8d00-505e17c95cf1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:35.922213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:35.922873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:35.923154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:35.923426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:35.924141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:35.926230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:35.956313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2293] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:35.957651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:35.968981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8962ca1-cc62-41c8-bdf0-d5707e40e3d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.029360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:36.037993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.038681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.038961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.039232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.039914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.041903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.062630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1153/2294] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:35 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:36.066990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.077294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92422aaa-4848-4049-a74b-07ca06f6a536 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.082775 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.083442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.083721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.083990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.084748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.086365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.097395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2295] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:36.103379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.114674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe3bd2-57df-4093-98c6-e26b7b8fb433 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.120079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.120976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.121258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.121525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] PATH_INFO: `/v3/projects/ce34d41981784bf0b65b145813183d2a/users/51937c755f944478b35f0e2d50ff05ce/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.138907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce34d41981784bf0b65b145813183d2a, user_id=51937c755f944478b35f0e2d50ff05ce, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.142500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.162202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1154/2296] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:36 2022] PUT /identity/v3/projects/ce34d41981784bf0b65b145813183d2a/users/51937c755f944478b35f0e2d50ff05ce/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:36.171124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.181971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93390d2e-527e-48e8-a61c-8e3f09ec3524 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.213459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:36.242864 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.243523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.243799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.244075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.244823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.246488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0b7fe-076b-4965-b3ff-b70bfdfe075c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.260673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2297] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:36.261986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.278788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b3bf2-d98a-49ae-9ef6-9783d8297138 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.299261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.300981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.301436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.301717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] PATH_INFO: `/v3/projects/ce34d41981784bf0b65b145813183d2a/users/51937c755f944478b35f0e2d50ff05ce/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.306539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce34d41981784bf0b65b145813183d2a, user_id=51937c755f944478b35f0e2d50ff05ce, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.312158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.327594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1155/2298] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:36 2022] PUT /identity/v3/projects/ce34d41981784bf0b65b145813183d2a/users/51937c755f944478b35f0e2d50ff05ce/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:36.339018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.339545 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.339933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.368441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] MFA Rules not processed for user `51937c755f944478b35f0e2d50ff05ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:36.369809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:36.445499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2299] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:36.461401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.463484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:36.536441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165c8a8-a943-4120-91cd-377b6c3c4d95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.540014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.540691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.540971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.541240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.551094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.552631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.562678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1156/2300] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:36.661383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:36.672926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e60ab-1c60-40f3-bfa9-6a5300927451 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:36.678022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:36.679371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:36.679717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:36.680120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:36.754847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:36.756358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d85b33-22dc-4e05-b3f9-f1b250455871 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:36.765989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2301] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:37.245522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:37.246804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:37.352182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e734e8f-d609-4771-bfc6-bae9fb02b298 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:37.357092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:37.357998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:37.358274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:37.358543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] PATH_INFO: `/v3/users/16fcb4a9fcbf4c3daca7ac1d3df7c8f3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:37.359782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16fcb4a9fcbf4c3daca7ac1d3df7c8f3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:37.361972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:37.444796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16fcb4a9fcbf4c3daca7ac1d3df7c8f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:37.449670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:37.458797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16fcb4a9fcbf4c3daca7ac1d3df7c8f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:37.459466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16fcb4a9fcbf4c3daca7ac1d3df7c8f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:37.474811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16fcb4a9fcbf4c3daca7ac1d3df7c8f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:37.477745 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16fcb4a9fcbf4c3daca7ac1d3df7c8f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:37.517329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1157/2302] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:37 2022] DELETE /identity/v3/users/16fcb4a9fcbf4c3daca7ac1d3df7c8f3 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:37.523760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6188-b981-4117-887a-527baa418a1a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:37.523760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6188-b981-4117-887a-527baa418a1a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:37.524364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d6188-b981-4117-887a-527baa418a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:37.570184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f40d6188-b981-4117-887a-527baa418a1a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:37.576147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f40d6188-b981-4117-887a-527baa418a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:37.694699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:37.722807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2303] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:37.733381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:37.735078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:37.886641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629d65f-ef17-4ec8-94ad-f23406116156 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:37.933391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:37.934311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:37.934627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:37.934916 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:37.963773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:37.966002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:38.005105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1158/2304] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:38.725372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d6188-b981-4117-887a-527baa418a1a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:38.727263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f40d6188-b981-4117-887a-527baa418a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:38.745681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d6188-b981-4117-887a-527baa418a1a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:38.753048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:38.754416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:38.754807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:38.755185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] PATH_INFO: `/v3/projects/a566b21e5554499ca67a34aed183bf09` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:38.756623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a566b21e5554499ca67a34aed183bf09)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:38.759134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:38.810896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a566b21e5554499ca67a34aed183bf09', 'request_id': 'req-5eb48d51-b325-4695-bc15-628b60ddd496'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:38.816451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a566b21e5554499ca67a34aed183bf09', 'request_id': 'req-5eb48d51-b325-4695-bc15-628b60ddd496'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:38.824927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a566b21e5554499ca67a34aed183bf09', 'request_id': 'req-5eb48d51-b325-4695-bc15-628b60ddd496'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:38.837524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] The token cache is being invalidate because project a566b21e5554499ca67a34aed183bf09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:38.837888 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:38.838509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5eb48d51-b325-4695-bc15-628b60ddd496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:38.840631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2305] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:38 2022] DELETE /identity/v3/projects/a566b21e5554499ca67a34aed183bf09 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:39.503973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:39.505428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:39.518377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5692b-d088-40a8-b890-d672137f774a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:39.593198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:39.594360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:39.594655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:39.594959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] PATH_INFO: `/v3/users/3641b99149e9486fa3777eb0b45c6085` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:39.596982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3641b99149e9486fa3777eb0b45c6085)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:39.599865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:39.652391 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3641b99149e9486fa3777eb0b45c6085'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:39.661881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3641b99149e9486fa3777eb0b45c6085'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:39.662795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3641b99149e9486fa3777eb0b45c6085'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:39.671980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3641b99149e9486fa3777eb0b45c6085'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:39.672816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3641b99149e9486fa3777eb0b45c6085'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:39.681689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:39.694256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1159/2306] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:39 2022] DELETE /identity/v3/users/3641b99149e9486fa3777eb0b45c6085 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:39.699033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:39.699428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:39.699783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:39.719204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:39.720674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:39.787820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2307] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:39.797692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:39.799337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:39.874586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae725a-763c-4e62-968f-185fa1c6ca9f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:39.878251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:39.879074 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:39.879346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:39.879619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:39.889510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:39.891042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:39.902293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1160/2308] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:40.181634 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:40.183122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:40.195573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1040959-ce5f-46df-acc9-bb3850925d0e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:40.201335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:40.202214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.202491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.202758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] PATH_INFO: `/v3/projects/5c6aa6527ffc4cfcadbc899f694919a3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.204113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6aa6527ffc4cfcadbc899f694919a3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:40.206062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:40.259152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6aa6527ffc4cfcadbc899f694919a3', 'request_id': 'req-33d0b36f-55e4-42fa-a60d-636c25019585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.264840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6aa6527ffc4cfcadbc899f694919a3', 'request_id': 'req-33d0b36f-55e4-42fa-a60d-636c25019585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.273644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6aa6527ffc4cfcadbc899f694919a3', 'request_id': 'req-33d0b36f-55e4-42fa-a60d-636c25019585'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.285795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] The token cache is being invalidate because project 5c6aa6527ffc4cfcadbc899f694919a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:40.286076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.286772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.288442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2309] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:40 2022] DELETE /identity/v3/projects/5c6aa6527ffc4cfcadbc899f694919a3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:40.293299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:40.304112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f2e1-00c8-47c0-9866-ec2aa2ca6430 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:40.315291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:40.317137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:40.325747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:40.387042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:40.388006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.388311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.388588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] PATH_INFO: `/v3/users/51937c755f944478b35f0e2d50ff05ce` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.389973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51937c755f944478b35f0e2d50ff05ce)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:40.393443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:40.401679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0b36f-55e4-42fa-a60d-636c25019585 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:40.407013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:40.408058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.408333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.408601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] PATH_INFO: `/v3/users/0436a3494d084bc599ad088ce477bd60` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.409952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0436a3494d084bc599ad088ce477bd60)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:40.412020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:40.467821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51937c755f944478b35f0e2d50ff05ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.477752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51937c755f944478b35f0e2d50ff05ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.478414 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0436a3494d084bc599ad088ce477bd60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.479870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51937c755f944478b35f0e2d50ff05ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.491439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51937c755f944478b35f0e2d50ff05ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.492706 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51937c755f944478b35f0e2d50ff05ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.494509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0436a3494d084bc599ad088ce477bd60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.495788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0436a3494d084bc599ad088ce477bd60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.496752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0436a3494d084bc599ad088ce477bd60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.516981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30b7f789-3f78-46ef-a7c3-886af59f3a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0436a3494d084bc599ad088ce477bd60'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.533605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1161/2310] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:40 2022] DELETE /identity/v3/users/51937c755f944478b35f0e2d50ff05ce => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:40.535584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2311] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:40 2022] DELETE /identity/v3/users/0436a3494d084bc599ad088ce477bd60 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:40.552454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.552764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.553017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.586714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:40.588360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:40.697037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2312] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:40.716768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:40.718342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:40.737911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:40.789395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a5374-3533-465c-ae72-c71142ae9a0e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:40.842601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:40.843268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.843557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.843838 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.857866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:40.858341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:40.859365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:40.859861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b97418-46c0-446b-b9dd-c2db8e72666e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:40.871846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1162/2313] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:40.872368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c9c8f-cb75-4b8a-8436-f68fc44c41ee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:40.877393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:40.879107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.879107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.879305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] PATH_INFO: `/v3/projects/ce34d41981784bf0b65b145813183d2a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:40.880445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce34d41981784bf0b65b145813183d2a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:40.882398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:40.904153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:40.947926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce34d41981784bf0b65b145813183d2a', 'request_id': 'req-56dba681-a25c-4a3b-88e4-b0b3662f6961'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.955029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce34d41981784bf0b65b145813183d2a', 'request_id': 'req-56dba681-a25c-4a3b-88e4-b0b3662f6961'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.964567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce34d41981784bf0b65b145813183d2a', 'request_id': 'req-56dba681-a25c-4a3b-88e4-b0b3662f6961'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.979843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] The token cache is being invalidate because project ce34d41981784bf0b65b145813183d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:40.980068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.981219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-56dba681-a25c-4a3b-88e4-b0b3662f6961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:40.986166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2314] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:40 2022] DELETE /identity/v3/projects/ce34d41981784bf0b65b145813183d2a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:40.987906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:40.988306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:40.988680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.009494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:41.010947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.084130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1163/2315] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:41.092876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.093467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.093811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.114341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:41.115572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.124107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2316] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:41.128830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.139654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaee4d6-a562-4320-ae5a-401ad426845e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.145309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.145982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.146258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.146530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.147260 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:41.147561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.149204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.163963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1164/2317] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:41.168783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.178882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17be4d-3959-46de-bbf7-4f56a006f29d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.184233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.184919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.185202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.185478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.186162 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.188113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.220660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2318] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:41.221821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.224059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.225114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.238910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8548c-cbb3-4f39-b65a-2e4ca1586e6c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.311868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.312878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.313156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.313429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] PATH_INFO: `/v3/projects/64e5d5f9b0014e6d8829737e139a5d1d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.314662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e5d5f9b0014e6d8829737e139a5d1d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.315070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad135-2d0d-484b-b40e-0a035c2e5352 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.316732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.318360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:41.320068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.320920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.321201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.321475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.322164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.324142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.346205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2319] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:41.350602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.362507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2c14b-2444-4f6b-941f-6b4704ca95a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.375582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e5d5f9b0014e6d8829737e139a5d1d', 'request_id': 'req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:41.384324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e5d5f9b0014e6d8829737e139a5d1d', 'request_id': 'req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:41.397868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e5d5f9b0014e6d8829737e139a5d1d', 'request_id': 'req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:41.412348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] The token cache is being invalidate because project 64e5d5f9b0014e6d8829737e139a5d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:41.412668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:41.413408 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4334b38c-d4a3-4c5c-8d14-9c3271cbe099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:41.415347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1165/2320] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:41 2022] DELETE /identity/v3/projects/64e5d5f9b0014e6d8829737e139a5d1d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:41.420379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6173cf-4c12-4033-acab-d6ae4fc649eb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.420896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6173cf-4c12-4033-acab-d6ae4fc649eb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.421124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6173cf-4c12-4033-acab-d6ae4fc649eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.428120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.428830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.429112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.429396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.430122 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.432617 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.444906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-da6173cf-4c12-4033-acab-d6ae4fc649eb None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:41.445453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2321] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:41.446919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-da6173cf-4c12-4033-acab-d6ae4fc649eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.449722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.451536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.460604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1166/2322] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:41.462247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:41.469668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433868-163a-4f85-af63-c18fa11464f2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.471362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.471736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.472249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.476376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.477443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.478059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.478331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.479055 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.480704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.492096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2323] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:41.492711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:41.494052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.497141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.504150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1167/2324] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:41.508443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.509252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7d164-d310-445c-ac5e-986c3567d075 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.513998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.514900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.515230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.515507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] PATH_INFO: `/v3/projects/7780e9307802426983f60a198c879c14/users/3bf18b67ae18480998d64c65b930d909/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.517670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17637fcd-5d43-411c-bdbe-284244283a18 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.522709 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.523376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.523660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.523935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.524739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:41.525033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.526961 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.531731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7780e9307802426983f60a198c879c14, user_id=3bf18b67ae18480998d64c65b930d909, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.534471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b12074d-e7ee-40d2-bf06-76a08176c174 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.539250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1168/2325] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:39:41.543481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.548564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2326] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:41 2022] PUT /identity/v3/projects/7780e9307802426983f60a198c879c14/users/3bf18b67ae18480998d64c65b930d909/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:41.553332 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae1b1a-0a96-41a3-a408-87c2875e091b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.558971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.559117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.559390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.588858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] MFA Rules not processed for user `3bf18b67ae18480998d64c65b930d909`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:41.590079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.613293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.613990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.614269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.614539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.615284 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.617206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.651205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1169/2327] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/projects => generated 361 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:41.652460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.668835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2328] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:41.679967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.681613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:41.717227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:41.744997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a261d-e01c-47a7-b03b-32d1920a2a84 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.750518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.750970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.751321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.751718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.753232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.756599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.779945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f57ba6-6192-47d6-9330-8834cd605f32 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.789685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1170/2329] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/users => generated 281 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:41.791063 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.809181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0bd93-8240-47fe-b27a-a3863beed605 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.815040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.816145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.816287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.816637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.817309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.818999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.835577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1171/2330] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:41.836581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.849952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a5138-08e8-40b0-a2b2-69a3acfa5c13 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.855226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.855905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.856238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.856510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.857309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.859265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.867068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.867739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.868015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.868315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.876533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1172/2331] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:41.881349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:41.903265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464563ef-db03-4ef6-83fc-394f80f14ef9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:41.903739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:41.915741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:41.917237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.920357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.920719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] PATH_INFO: `/v3/projects/785c4bc2c0f84963b88554f4d9142241/users/13f56faa54d64ca8b4539b38bb71dc86/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:41.945792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.947781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ee1f9-d6e3-4271-afda-f4ea916d2608 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.953025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] RBAC: Authorizing `identity:create_grant(project_id=785c4bc2c0f84963b88554f4d9142241, user_id=13f56faa54d64ca8b4539b38bb71dc86, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:41.956568 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:41.958095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2332] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:39:41.977789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1173/2333] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:41 2022] PUT /identity/v3/projects/785c4bc2c0f84963b88554f4d9142241/users/13f56faa54d64ca8b4539b38bb71dc86/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:41.988194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:41.988837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:41.989513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:42.023407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] MFA Rules not processed for user `13f56faa54d64ca8b4539b38bb71dc86`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:42.025365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:42.135716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2334] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:42.169157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:42.171370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:42.236902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223523b3-084c-4539-8678-9286dbde64da admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:42.241318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:42.242180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:42.242463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:42.242740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:42.252361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:42.255112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96262fac-42e6-4e36-b9a4-996345260ee9 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:42.266253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1174/2335] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:42.281495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:42.392621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22728b89-cbc9-4486-945e-48116a87a55f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:42.398696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:42.399735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:42.400335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:42.400678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] PATH_INFO: `/v3/users/13f56faa54d64ca8b4539b38bb71dc86` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:42.401918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f56faa54d64ca8b4539b38bb71dc86)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:42.404067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:42.463525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f56faa54d64ca8b4539b38bb71dc86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:42.471528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f56faa54d64ca8b4539b38bb71dc86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:42.472486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f56faa54d64ca8b4539b38bb71dc86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:42.473646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f56faa54d64ca8b4539b38bb71dc86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:42.478707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:42.488705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f56faa54d64ca8b4539b38bb71dc86'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:42.583628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2336] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:42 2022] DELETE /identity/v3/users/13f56faa54d64ca8b4539b38bb71dc86 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:42.583829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:42.583829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:42.583829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:42.583829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:42.583829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:42.641963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1175/2337] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:42.654238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:42.655898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:42.730733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d21c-fa1b-4a2b-8108-6108e57a2399 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:42.734325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:42.734980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:42.735260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:42.735534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:42.745873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:42.747349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50672e2-3446-4f8d-9e05-e8c6eeee5984 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:42.757305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2338] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:43.054207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.055800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:43.069613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f3de2-4efb-45e8-b88f-08c7f2939b32 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.075216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.076123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.076409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.076681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] PATH_INFO: `/v3/projects/785c4bc2c0f84963b88554f4d9142241` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.077923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=785c4bc2c0f84963b88554f4d9142241)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.079805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.121321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:43.139122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785c4bc2c0f84963b88554f4d9142241', 'request_id': 'req-c70f9634-dd01-42a9-8e4e-075f6b679e5a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:43.145856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785c4bc2c0f84963b88554f4d9142241', 'request_id': 'req-c70f9634-dd01-42a9-8e4e-075f6b679e5a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:43.156136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785c4bc2c0f84963b88554f4d9142241', 'request_id': 'req-c70f9634-dd01-42a9-8e4e-075f6b679e5a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:43.177680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] The token cache is being invalidate because project 785c4bc2c0f84963b88554f4d9142241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:43.177964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:43.178620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c70f9634-dd01-42a9-8e4e-075f6b679e5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:43.180252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1176/2339] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:43 2022] DELETE /identity/v3/projects/785c4bc2c0f84963b88554f4d9142241 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:43.185210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.185664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.185764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.204560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:43.205780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:43.277724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2340] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:43.285782 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.286081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.286353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.309032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:43.310341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:43.318346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1177/2341] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:43.322678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.334428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013e3a-7f82-40aa-aa8f-64c029938bf7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.339654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.340458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.340796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.341079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.341800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:43.342100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.343727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.360484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2342] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:43.365087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.378090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbad23-0e96-4eda-96af-f1579468000f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.384847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.385872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.386336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.386677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.391691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.397661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.438993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1178/2343] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:43.443305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.465693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d621f-72e3-478d-8955-b5a45db0fbc8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.531543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.532249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.532554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.532823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.533540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.535686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.545318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:43.561504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2344] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:43.562765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.574337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935de0c-05f9-4bd6-8fb5-1c64fe28814a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.579111 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.579779 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.580091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.580366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.581101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.582764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.595081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1179/2345] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:43.599498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.609604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3fa3a-fde6-441f-9b51-64f9196fd6ee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.614488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.615153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.615434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.615709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.616469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.618112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171eef66-8827-4dd9-ba59-5155018220cd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.628481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2346] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:43.633186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.649394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e0a23-7dd9-48da-bef3-b0bdd407e55b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.657888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.659215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.659619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.659828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] PATH_INFO: `/v3/projects/e738b81f80ad4be1ac855d1b39c7506f/users/29bb33e9c8af4101be144405802bcb7b/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.676679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e738b81f80ad4be1ac855d1b39c7506f, user_id=29bb33e9c8af4101be144405802bcb7b, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.679350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.696918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1180/2347] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:43 2022] PUT /identity/v3/projects/e738b81f80ad4be1ac855d1b39c7506f/users/29bb33e9c8af4101be144405802bcb7b/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:43.707870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.708188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.708466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.739685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] MFA Rules not processed for user `29bb33e9c8af4101be144405802bcb7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:43.740978 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:43.820101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2348] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:39:43.824885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.888877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ff276-f03b-485e-bf03-28a9677ae830 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:43.893674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:43.894699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:43.895129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:43.895508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:43.896235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:43.898148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:43.930833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1181/2349] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:43.934897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:43.951127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088eaa-a90f-4b78-94ed-ebaeed2a538e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.036318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.037799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.038141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.038498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.039556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.045193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.071420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2350] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:43 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:44.074935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.091628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3ceaa-43e1-4b71-ae8d-97c5ed92ea76 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.098812 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.099448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.099727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.100120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.100750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.102441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.114727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1182/2351] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:44.118792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.133310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925baba-fd31-4c02-8ed5-4af89b10720d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.138413 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.139276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.139490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.139763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.140521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.142143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b1f6c-9e9e-4d0c-80f6-27fed3500203 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.156351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2352] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:44.158974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.175254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882833d1-4aa4-4e23-b059-19847ebd10fb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.180823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.181927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.182206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.182482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] PATH_INFO: `/v3/projects/4977d32ab900446d998d7a846b9c224d/users/55b0e7d2ea934b3b8c8cd8d11bc01ad4/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.198130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4977d32ab900446d998d7a846b9c224d, user_id=55b0e7d2ea934b3b8c8cd8d11bc01ad4, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.201221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.215774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1183/2353] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:44 2022] PUT /identity/v3/projects/4977d32ab900446d998d7a846b9c224d/users/55b0e7d2ea934b3b8c8cd8d11bc01ad4/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:44.226486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.227014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.227014 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.257842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] MFA Rules not processed for user `55b0e7d2ea934b3b8c8cd8d11bc01ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:44.259069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:44.309860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:44.335113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2354] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:44.339829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.346230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:44.410897 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b79d61-c7c0-4c42-80c5-d76349555e5f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.415648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.416335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.416616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.417015 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.417781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.419958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.447662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1184/2355] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:44 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:44.451789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.463615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a68b8-699d-4e66-9381-33ffd00344ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.528518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.530271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.530638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.531033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.531804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.533826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.560517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2356] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:44 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:44.564724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.576003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08bff4-d257-4a06-8614-1cced313e7b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.585762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.588162 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.588551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.588950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.589787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.591786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.604273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1185/2357] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:44.608776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.619557 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95795a0d-5332-4ace-840d-8d00e75ef000 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.624791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.626563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.626845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.627115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] PATH_INFO: `/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a/users/087e984d825d47e9ad0531f542f758a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.645325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe538bdde0a4953bb43fc72537c5f2a, user_id=087e984d825d47e9ad0531f542f758a0, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.648467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.669452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2358] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:44 2022] PUT /identity/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a/users/087e984d825d47e9ad0531f542f758a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:39:44.673614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.685735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cee48ab-4bfc-4fb5-b863-f9a0617951a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.760070 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.760749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.761033 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.761307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.762043 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.763721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf266ad-61a0-46b4-9fcb-9a3f9fcf789e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.784695 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1186/2359] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:44.789455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.800441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07068f8c-653a-4ada-8f74-970d09cbf0ec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:44.806138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:44.807134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.807485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.807762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] PATH_INFO: `/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a/users/087e984d825d47e9ad0531f542f758a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.809812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffe538bdde0a4953bb43fc72537c5f2a, user_id=087e984d825d47e9ad0531f542f758a0, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:44.812595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:44.829907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2360] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:44 2022] PUT /identity/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a/users/087e984d825d47e9ad0531f542f758a0/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:44.840512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:44.840808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:44.841086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:44.877154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] MFA Rules not processed for user `087e984d825d47e9ad0531f542f758a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:44.878628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:44.954932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:44.965974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1187/2361] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:44.970964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:44.973018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:45.037897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b69c-cf4d-46c3-9d37-68fa846ac551 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:45.043504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:45.044229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:45.044521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:45.044788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:45.104951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:45.116730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:45.118388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:45.129579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2362] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:46.657452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:46.659210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:46.674484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1491e-c6f0-4db8-895b-a21d919575ae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:46.678630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:46.679435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:46.679717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:46.679988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:46.738647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:46.740215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:46.750581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1188/2363] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:47.565094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:47.576936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ae6e-ab0b-4101-b81e-b503fded3f77 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:47.582872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:47.584240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:47.584573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:47.584854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:47.603214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:47.605022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:47.619554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2364] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:47.895165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:47.897714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:47.972685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babff40-ce90-425a-85ac-72b97d8aefa8 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:47.977336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:47.978263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:47.978541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:47.978810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] PATH_INFO: `/v3/users/3847ea2eff434d1f95a744cda1ec24a9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:47.979972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3847ea2eff434d1f95a744cda1ec24a9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:47.981919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:48.029389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3847ea2eff434d1f95a744cda1ec24a9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.038174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3847ea2eff434d1f95a744cda1ec24a9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.038783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3847ea2eff434d1f95a744cda1ec24a9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.046689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3847ea2eff434d1f95a744cda1ec24a9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.047368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3847ea2eff434d1f95a744cda1ec24a9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.064808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1189/2365] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:47 2022] DELETE /identity/v3/users/3847ea2eff434d1f95a744cda1ec24a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:48.247332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:48.261154 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5e86f-e3b1-498f-b515-7a0f8a4265b6 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:48.331626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:48.331626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:48.331626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:48.331626 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] PATH_INFO: `/v3/projects/28eaf8647df449ca9c9a1cd8b1cfaa6e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:48.332808 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28eaf8647df449ca9c9a1cd8b1cfaa6e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:48.334820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:48.386927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28eaf8647df449ca9c9a1cd8b1cfaa6e', 'request_id': 'req-156dfa5c-077b-43ca-997f-407a5f42bf6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.392715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28eaf8647df449ca9c9a1cd8b1cfaa6e', 'request_id': 'req-156dfa5c-077b-43ca-997f-407a5f42bf6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.401078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28eaf8647df449ca9c9a1cd8b1cfaa6e', 'request_id': 'req-156dfa5c-077b-43ca-997f-407a5f42bf6f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.411937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] The token cache is being invalidate because project 28eaf8647df449ca9c9a1cd8b1cfaa6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:48.412246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.412851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.414376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2366] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:48 2022] DELETE /identity/v3/projects/28eaf8647df449ca9c9a1cd8b1cfaa6e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:48.418496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:48.419906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:48.433553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c343f9a-ee1e-4f08-af64-f878ca499314 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:48.446678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:48.493518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:48.494494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:48.494788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:48.495056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] PATH_INFO: `/v3/users/8ad193b8e98948e58932bc4e39e5e9ca` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:48.496360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad193b8e98948e58932bc4e39e5e9ca)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:48.498780 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:48.551354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad193b8e98948e58932bc4e39e5e9ca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.559655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad193b8e98948e58932bc4e39e5e9ca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.560322 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad193b8e98948e58932bc4e39e5e9ca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.567275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad193b8e98948e58932bc4e39e5e9ca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.567942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d5875bd6-fc92-4d12-b7e3-a81cd9d334ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad193b8e98948e58932bc4e39e5e9ca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.586833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1190/2367] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:48 2022] DELETE /identity/v3/users/8ad193b8e98948e58932bc4e39e5e9ca => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:48.770300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:48.770300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:48.782745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156dfa5c-077b-43ca-997f-407a5f42bf6f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:48.841960 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:48.842859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:48.843139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:48.843415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] PATH_INFO: `/v3/projects/f9bde3047af445608f3436d09ae05561` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:48.844623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bde3047af445608f3436d09ae05561)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:48.846524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:48.854684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:48.897256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bde3047af445608f3436d09ae05561', 'request_id': 'req-04c47c0b-2aae-4864-bbe3-57d2a6a81215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.903794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bde3047af445608f3436d09ae05561', 'request_id': 'req-04c47c0b-2aae-4864-bbe3-57d2a6a81215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.912330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bde3047af445608f3436d09ae05561', 'request_id': 'req-04c47c0b-2aae-4864-bbe3-57d2a6a81215'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.923919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] The token cache is being invalidate because project f9bde3047af445608f3436d09ae05561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:48.924321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.924985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-04c47c0b-2aae-4864-bbe3-57d2a6a81215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:48.926535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2368] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:48 2022] DELETE /identity/v3/projects/f9bde3047af445608f3436d09ae05561 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:48.931417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:48.931934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:48.932058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:48.950882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:48.952144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:49.040217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1191/2369] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:49.041717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.042007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.042283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.066576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:49.071096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:49.086763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2370] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:49.090753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.101816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22440e8-057c-4622-b929-d2cda9b8f5a7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.107268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.108570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.109078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.109194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.109852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:39:49.110159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.111990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.129513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1192/2371] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:39:49.130589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.144544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5a7d1-5c87-4170-bc0d-f2c760d8d881 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.145429 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.146208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.146563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.146922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.147707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.149743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.183070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2372] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:49.185325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.195369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a56661-ac7e-4e7b-8800-19129d20965a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.215227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:49.252981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.253761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.254046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.254323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.255017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.256992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.282977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1193/2373] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:49.284236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.296019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97bbd7-9942-453b-ba96-d00381db4f2f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.303968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.305120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.305552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.306021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.306782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.308466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.320434 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2374] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:39:49.324711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.335872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71eb156-4e93-4ef5-99b8-ee611d496236 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.341880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.342616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.342830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.343096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.343828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.345497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f74fb-d9ea-4e80-93cd-306fa60c928e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.356055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1194/2375] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:49.360325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.370899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f3c87-65ff-4239-ba91-b2e0005e1541 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.376366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.377302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.377596 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.377869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] PATH_INFO: `/v3/projects/5d3330f53cc6442bacc7717218837803/users/9254370db04b4c5e85c700f8b2f9f730/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.393597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3330f53cc6442bacc7717218837803, user_id=9254370db04b4c5e85c700f8b2f9f730, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.396457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.411573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2376] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:49 2022] PUT /identity/v3/projects/5d3330f53cc6442bacc7717218837803/users/9254370db04b4c5e85c700f8b2f9f730/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:49.422249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.422539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.422816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.450369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] MFA Rules not processed for user `9254370db04b4c5e85c700f8b2f9f730`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:49.451580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:49.520851 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1195/2377] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:49.525874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.590485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9dc78-a278-4102-aee3-577964974013 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.595678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.596311 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.596589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.596973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.597689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.599604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.626392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2378] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:39:49.630678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.641463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fc551-b261-48e8-a990-fdb4ed012a54 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.700755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.701425 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.701700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.701970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.702804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.704776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.728352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1196/2379] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:39:49.732832 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.749335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b4d0f-9ff5-4101-b9fa-564be14a240f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.756441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.757106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.757386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.757656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.758382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.759990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.771143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2380] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:49.775723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.786786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bcfb8-aaa4-41f4-80f4-bd2b8feccf8e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.791896 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.792820 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.793099 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.793361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] PATH_INFO: `/v3/projects/a2b2d12413464660adb9752bd6264917/users/946d12d315de4e258ed27286f005a6d8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.808770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b2d12413464660adb9752bd6264917, user_id=946d12d315de4e258ed27286f005a6d8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.811410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.820732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:49.826546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1197/2381] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:49 2022] PUT /identity/v3/projects/a2b2d12413464660adb9752bd6264917/users/946d12d315de4e258ed27286f005a6d8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:49.830666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.840797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd48ec-bb4d-4d54-b376-59b551cfebee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.844467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:49.900390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.901058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.901328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.901594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.902305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.903911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ae30d-0c70-426b-8a51-610e73e2cadf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.913718 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2382] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:39:49.918139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:49.929303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c51b6a-2a7a-4250-9543-00436d6d8f2b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:49.934663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:49.935556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.935834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.936137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] PATH_INFO: `/v3/projects/a2b2d12413464660adb9752bd6264917/users/946d12d315de4e258ed27286f005a6d8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.938184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b2d12413464660adb9752bd6264917, user_id=946d12d315de4e258ed27286f005a6d8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:49.940923 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:49.951460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1198/2383] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:39:49 2022] PUT /identity/v3/projects/a2b2d12413464660adb9752bd6264917/users/946d12d315de4e258ed27286f005a6d8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:49.962649 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:49.962943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:49.963215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:49.990880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] MFA Rules not processed for user `946d12d315de4e258ed27286f005a6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:49.992045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:50.065886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2384] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:50.077294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:50.079670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:50.142409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051e351-2b4f-4adc-b2e4-09b72f700732 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:50.146364 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:50.147144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:50.147407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:50.147692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:50.209656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:50.211504 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b70d8-d335-4247-a0a5-b7bcf77fafcf service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:50.221843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1199/2385] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:54.725107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:54.726603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:54.795752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48989dbb-26ca-45c8-8673-57b930357429 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:54.802656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:54.803547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:54.803823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:54.804126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] PATH_INFO: `/v3/users/29bb33e9c8af4101be144405802bcb7b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:54.805422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bb33e9c8af4101be144405802bcb7b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:54.807421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:54.845202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:54.858884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bb33e9c8af4101be144405802bcb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:54.866796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bb33e9c8af4101be144405802bcb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:54.867416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bb33e9c8af4101be144405802bcb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:54.868240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bb33e9c8af4101be144405802bcb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:54.884422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bb33e9c8af4101be144405802bcb7b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:54.897335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2386] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:54 2022] DELETE /identity/v3/users/29bb33e9c8af4101be144405802bcb7b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:54.902397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:54.902892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:54.902983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:54.922521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:39:54.923727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:54.983590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1200/2387] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:39:54.993799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:54.995271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:55.052738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa477-b82f-4d1b-8803-7e69955d70ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:55.056589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:55.057234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:55.057495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:55.057756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:55.066837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:55.068314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:55.078206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2388] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:39:55.355128 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:55.357115 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:55.377184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6faa-6257-4239-98ef-70160aaef3b1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:55.382189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:55.383079 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:55.383359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:55.383642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] PATH_INFO: `/v3/projects/e738b81f80ad4be1ac855d1b39c7506f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:55.384876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e738b81f80ad4be1ac855d1b39c7506f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:55.387117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:55.430327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:55.443060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e738b81f80ad4be1ac855d1b39c7506f', 'request_id': 'req-4f998bd2-6e5e-4157-9ba4-2d444059a551'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.449377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e738b81f80ad4be1ac855d1b39c7506f', 'request_id': 'req-4f998bd2-6e5e-4157-9ba4-2d444059a551'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.458545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e738b81f80ad4be1ac855d1b39c7506f', 'request_id': 'req-4f998bd2-6e5e-4157-9ba4-2d444059a551'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.470114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] The token cache is being invalidate because project e738b81f80ad4be1ac855d1b39c7506f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:55.470513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.471140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.474447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1201/2389] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:55 2022] DELETE /identity/v3/projects/e738b81f80ad4be1ac855d1b39c7506f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:55.476921 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:55.486555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005a896-ee44-4b99-a982-81c6be61070d service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:55.557296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:55.558188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:55.558473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:55.558747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] PATH_INFO: `/v3/users/55b0e7d2ea934b3b8c8cd8d11bc01ad4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:55.559930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55b0e7d2ea934b3b8c8cd8d11bc01ad4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:55.561923 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:55.613369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55b0e7d2ea934b3b8c8cd8d11bc01ad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.617651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:55.624578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55b0e7d2ea934b3b8c8cd8d11bc01ad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.625188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55b0e7d2ea934b3b8c8cd8d11bc01ad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.626084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55b0e7d2ea934b3b8c8cd8d11bc01ad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.647496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55b0e7d2ea934b3b8c8cd8d11bc01ad4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:55.687701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2390] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:55 2022] DELETE /identity/v3/users/55b0e7d2ea934b3b8c8cd8d11bc01ad4 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:56.108265 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:56.110346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:56.123836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998bd2-6e5e-4157-9ba4-2d444059a551 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:56.182055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:56.182931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:56.183217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:56.183488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] PATH_INFO: `/v3/projects/4977d32ab900446d998d7a846b9c224d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:56.184669 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4977d32ab900446d998d7a846b9c224d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:56.186601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:56.242630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4977d32ab900446d998d7a846b9c224d', 'request_id': 'req-bd25181d-795b-421b-80dc-72b564f7c907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.248482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4977d32ab900446d998d7a846b9c224d', 'request_id': 'req-bd25181d-795b-421b-80dc-72b564f7c907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.259253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4977d32ab900446d998d7a846b9c224d', 'request_id': 'req-bd25181d-795b-421b-80dc-72b564f7c907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.270281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] The token cache is being invalidate because project 4977d32ab900446d998d7a846b9c224d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:56.270562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.271192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.272801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1202/2391] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:56 2022] DELETE /identity/v3/projects/4977d32ab900446d998d7a846b9c224d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:56.277218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:56.278696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:56.291268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f83dee-e1d7-4623-a0fa-549141438f61 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:56.348509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:56.349394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:56.349669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:56.349942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] PATH_INFO: `/v3/users/087e984d825d47e9ad0531f542f758a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:56.351137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=087e984d825d47e9ad0531f542f758a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:56.353116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:56.404492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087e984d825d47e9ad0531f542f758a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.413318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087e984d825d47e9ad0531f542f758a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.413961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087e984d825d47e9ad0531f542f758a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.414671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087e984d825d47e9ad0531f542f758a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.427073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087e984d825d47e9ad0531f542f758a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.439717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2392] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:39:56 2022] DELETE /identity/v3/users/087e984d825d47e9ad0531f542f758a0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:56.719558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:56.721036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:56.744298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd25181d-795b-421b-80dc-72b564f7c907 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:56.795064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:56.807657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:56.808752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:56.809256 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:56.809371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] PATH_INFO: `/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:56.810771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffe538bdde0a4953bb43fc72537c5f2a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:56.812913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:56.862236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffe538bdde0a4953bb43fc72537c5f2a', 'request_id': 'req-45a207f1-86bd-40c9-83bb-35030a2c4133'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.867966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffe538bdde0a4953bb43fc72537c5f2a', 'request_id': 'req-45a207f1-86bd-40c9-83bb-35030a2c4133'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.877608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffe538bdde0a4953bb43fc72537c5f2a', 'request_id': 'req-45a207f1-86bd-40c9-83bb-35030a2c4133'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.894826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] The token cache is being invalidate because project ffe538bdde0a4953bb43fc72537c5f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:39:56.895103 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.895789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-45a207f1-86bd-40c9-83bb-35030a2c4133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:39:56.897533 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1203/2393] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:39:56 2022] DELETE /identity/v3/projects/ffe538bdde0a4953bb43fc72537c5f2a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:39:58.990770 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:39:58.992486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:59.056636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:39:59.061039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448ac96c-f163-48de-b67e-3ad13509fc25 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:39:59.116234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:39:59.116907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:39:59.117201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:39:59.117512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:39:59.119513 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:39:59.186882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:39:59.188299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:39:59.198957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2394] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:39:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:02.160433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1d465-7bd5-4e1e-9b2e-eb19b0d66a3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:02.160945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1d465-7bd5-4e1e-9b2e-eb19b0d66a3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:02.161130 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1d465-7bd5-4e1e-9b2e-eb19b0d66a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:02.191399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3bd1d465-7bd5-4e1e-9b2e-eb19b0d66a3b None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:02.192743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bd1d465-7bd5-4e1e-9b2e-eb19b0d66a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:02.276218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1204/2395] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:40:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:02.321929 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:02.323436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:02.451585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0f258-e459-444b-b670-4965c30c840b service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:02.459442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:02.460252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:02.460992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:02.460992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:02.475847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:02.480172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:02.482523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:02.491753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2396] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:40:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:40:06.020909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797f8ca-0caa-409e-974f-74e935fd2437 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:06.025742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797f8ca-0caa-409e-974f-74e935fd2437 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:06.026109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797f8ca-0caa-409e-974f-74e935fd2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:06.070622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b797f8ca-0caa-409e-974f-74e935fd2437 None None] MFA Rules not processed for user `6d07884e92bc4cfe95d2fd99c40f25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:06.072388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b797f8ca-0caa-409e-974f-74e935fd2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:06.151509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1205/2397] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:06.316807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:06.318956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:06.428220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b88a0c-774b-403e-bf30-e477852bf297 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:06.437492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:06.438516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:06.438885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:06.439160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] PATH_INFO: `/v3/users/9254370db04b4c5e85c700f8b2f9f730` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:06.440415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9254370db04b4c5e85c700f8b2f9f730)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:06.442370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:06.497781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9254370db04b4c5e85c700f8b2f9f730'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:06.509561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9254370db04b4c5e85c700f8b2f9f730'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:06.510720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9254370db04b4c5e85c700f8b2f9f730'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:06.511320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9254370db04b4c5e85c700f8b2f9f730'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:06.526460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9254370db04b4c5e85c700f8b2f9f730'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:06.542326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2398] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:06 2022] DELETE /identity/v3/users/9254370db04b4c5e85c700f8b2f9f730 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:06.547017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:06.547304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:06.547576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:06.556611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:06.585041 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:06.586441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:06.657963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1206/2399] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:06.673325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:06.677419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:06.764464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:06.807816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db5348-36b5-45e0-aeaf-7d263a203fdb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:06.812698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:06.813591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:06.813964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:06.814443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:06.826453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:06.830650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:06.841501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2400] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:07.201397 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:07.202761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:07.222821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbdff6-43a6-4c2a-bf27-591384cc8c3c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:07.254283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:07.257362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:07.258540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:07.260083 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] PATH_INFO: `/v3/projects/5d3330f53cc6442bacc7717218837803` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:07.265283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3330f53cc6442bacc7717218837803)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:07.271152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:07.498974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3330f53cc6442bacc7717218837803', 'request_id': 'req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.518431 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3330f53cc6442bacc7717218837803', 'request_id': 'req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.539562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3330f53cc6442bacc7717218837803', 'request_id': 'req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.553955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] The token cache is being invalidate because project 5d3330f53cc6442bacc7717218837803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:07.554270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.561795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.563456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1207/2401] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:07 2022] DELETE /identity/v3/projects/5d3330f53cc6442bacc7717218837803 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:07.567585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:07.577663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67d5f9-4b1f-41f2-978e-e4fb0914004f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:07.637821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:07.638934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:07.639210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:07.639480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] PATH_INFO: `/v3/users/946d12d315de4e258ed27286f005a6d8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:07.640732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946d12d315de4e258ed27286f005a6d8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:07.642679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:07.695896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946d12d315de4e258ed27286f005a6d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.704323 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946d12d315de4e258ed27286f005a6d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.704976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946d12d315de4e258ed27286f005a6d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.705593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946d12d315de4e258ed27286f005a6d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.718542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e969deab-2625-4073-ae21-2e2242bb8e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946d12d315de4e258ed27286f005a6d8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:07.733586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2402] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:07 2022] DELETE /identity/v3/users/946d12d315de4e258ed27286f005a6d8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:08.058413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.060101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.074468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e11c5-5c07-41f0-8163-a0cdc7f483a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.154306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.155191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.155465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.155738 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] PATH_INFO: `/v3/projects/a2b2d12413464660adb9752bd6264917` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.156955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b2d12413464660adb9752bd6264917)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.158891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.186276 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:08.216824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b2d12413464660adb9752bd6264917', 'request_id': 'req-6f1f8e61-6665-4c31-8b01-253033927c5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.222540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b2d12413464660adb9752bd6264917', 'request_id': 'req-6f1f8e61-6665-4c31-8b01-253033927c5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.231016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b2d12413464660adb9752bd6264917', 'request_id': 'req-6f1f8e61-6665-4c31-8b01-253033927c5f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.243341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] The token cache is being invalidate because project a2b2d12413464660adb9752bd6264917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:08.243871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.244975 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6f1f8e61-6665-4c31-8b01-253033927c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.246724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1208/2403] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:08 2022] DELETE /identity/v3/projects/a2b2d12413464660adb9752bd6264917 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:08.253805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.254344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.254799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.302399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:08.303838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.339318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:08.377707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2404] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:08.387808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.388104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.388357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.416305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:08.418315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.432266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1209/2405] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:08.434338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.445449 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa1e3f-9ed5-432e-93ab-f96f89b4beb2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.458002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.458587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.459179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.459302 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.460007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:40:08.460340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.461984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.480064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2406] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:40:08.482184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.491597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f133-e364-456e-832e-3838d14b25b2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.496463 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.497327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.497735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.500683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.501723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.505630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.538094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1210/2407] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:08 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:40:08.543240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.566576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e0d-7182-4a83-b5ba-cd040431b40d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.582085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.585736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.669020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8c9e9-ab05-484e-b75f-7bfdac0c62eb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.674163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.675346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.675698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.676082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] PATH_INFO: `/v3/users/30756eb5a7ac4f2683c9818ebde03553` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.677500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30756eb5a7ac4f2683c9818ebde03553)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.679774 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.679892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.680720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.680906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.681180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.681873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.684695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.731310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2408] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:08 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:08.738092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.759898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9539346-d870-422b-90f8-8baef558cb55 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.770875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.772607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30756eb5a7ac4f2683c9818ebde03553'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.772997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.773951 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.774729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.776504 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.789352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.793657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30756eb5a7ac4f2683c9818ebde03553'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.795634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30756eb5a7ac4f2683c9818ebde03553'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.805173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2409] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:40:08.806212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.808166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.811027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30756eb5a7ac4f2683c9818ebde03553'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.811793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f74d2927-8928-4504-bf1d-fcf1a68fc564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30756eb5a7ac4f2683c9818ebde03553'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:08.823594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f43d90-97c0-42ee-a77b-16c38469ad95 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.834094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.835248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.835743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.836239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.837223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.839203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:08.844238 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1211/2410] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:08 2022] DELETE /identity/v3/users/30756eb5a7ac4f2683c9818ebde03553 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:08.848677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.848965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.849370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.855206 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:08.858232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2411] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:08.862920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.874937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:08.876298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.879306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797d6ba-4236-4ba9-81d5-ceea16c9d9f9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:08.951047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1212/2412] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:08.962625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:08.965006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:08.971136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:08.973256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:08.973814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:08.974308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] PATH_INFO: `/v3/projects/9c1a34b570ee4757a87ae6fad5dc234e/users/6d3e2599cbca4ac78d8d53927b881c1f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:08.995720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1a34b570ee4757a87ae6fad5dc234e, user_id=6d3e2599cbca4ac78d8d53927b881c1f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:08.999180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3923d5-7c01-40da-9366-37f134694920 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:09.019801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2413] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:08 2022] PUT /identity/v3/projects/9c1a34b570ee4757a87ae6fad5dc234e/users/6d3e2599cbca4ac78d8d53927b881c1f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:09.032189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.032788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.033279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.046005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a80a9-842a-40d6-95d1-27eda235a821 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:09.057534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:09.067657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] MFA Rules not processed for user `6d3e2599cbca4ac78d8d53927b881c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:09.069791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:09.125048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:09.125964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.126243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.126515 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.148280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2414] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:09.164381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:09.166223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:09.216893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:09.218914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:09.234705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1213/2415] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:09.251814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139948dd-d1e2-4e63-b8c4-a34a8cfe265a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:09.256728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:09.257400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.257677 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.257952 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.267781 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:09.269473 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a86cb5-bf47-4ec1-b17a-48f8f698b5f3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:09.280428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2416] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:40:09.599658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:09.604014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:09.637559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a32144-ba49-4130-912a-d152330e9319 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:09.650678 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:09.651596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.651878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.652327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] PATH_INFO: `/v3/projects/6dfc8a42259240f091a60df654a51129` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.653986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dfc8a42259240f091a60df654a51129)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:09.656384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:09.659282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:09.731689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dfc8a42259240f091a60df654a51129', 'request_id': 'req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:09.744699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dfc8a42259240f091a60df654a51129', 'request_id': 'req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:09.763567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dfc8a42259240f091a60df654a51129', 'request_id': 'req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:09.811040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] The token cache is being invalidate because project 6dfc8a42259240f091a60df654a51129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:09.811323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:09.812417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec182687-5a17-4fbe-aeb6-bbaa6a20998c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:09.823632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1214/2417] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:09 2022] DELETE /identity/v3/projects/6dfc8a42259240f091a60df654a51129 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:09.825685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.826192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.826281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.874881 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:09.876171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:09.968079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2418] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:09.973040 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:09.973347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:09.974227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:09.999574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:10.000963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:10.010518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1215/2419] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:10.015226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.030905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d9696-60a9-4096-b7ba-600ec398f4ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.036478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.037356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.037659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.038428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.038662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:40:10.038973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.040648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.057371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2420] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:40:10.061873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.072847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b400ddd-0e37-4643-85ce-46af943710e2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.084244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.085668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.086235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.086643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.087351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.089870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.126498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1216/2421] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:10 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:10.130831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.145880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9129e-a627-4385-8209-f30259c27352 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.178620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:10.209170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.210004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.210382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.210984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.211773 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.213872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.238304 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2422] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:10 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:10.243241 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.256685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0db7e2-fe69-4088-8ba6-4259ddc418c4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.264021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.264740 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.265027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.265303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.266042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.267700 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.280423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1217/2423] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:10.284499 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.294815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf205-77fd-4aaf-aa6d-045918e2c6ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.299556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.300245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.300531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.300810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.301543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.303553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ed13-a5df-4ed2-b813-ae5242c8dc94 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.313893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2424] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:10.318366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.329717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71bcb3-8ec2-41d6-a4e2-43e4eca61517 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.337444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.338690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.339226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.339693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] PATH_INFO: `/v3/projects/934272ce955f46d489f97a6a4179791a/users/5f2c10ac03fd41e7a595ee63e4a4789b/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.356000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934272ce955f46d489f97a6a4179791a, user_id=5f2c10ac03fd41e7a595ee63e4a4789b, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.358767 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.372323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1218/2425] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:10 2022] PUT /identity/v3/projects/934272ce955f46d489f97a6a4179791a/users/5f2c10ac03fd41e7a595ee63e4a4789b/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:40:10.382542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d500-8263-442d-acaa-0130299477ea None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.382837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d500-8263-442d-acaa-0130299477ea None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.383119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d500-8263-442d-acaa-0130299477ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.410581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1824d500-8263-442d-acaa-0130299477ea None None] MFA Rules not processed for user `5f2c10ac03fd41e7a595ee63e4a4789b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:10.411989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1824d500-8263-442d-acaa-0130299477ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:10.485160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2426] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:10.505167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:10.507772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:10.623964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf13db-a5fc-4c7c-ac5c-97505a4c8095 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:10.631148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:10.633075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:10.633439 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:10.634326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:10.681273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:10.682798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc952737-977b-4b46-af60-f9585cee3a5c service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:10.711675 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1219/2427] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:20.481556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:20.706489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:21.704481 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824d500-8263-442d-acaa-0130299477ea None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:21.705693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1824d500-8263-442d-acaa-0130299477ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:21.788021 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824d500-8263-442d-acaa-0130299477ea None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:21.793050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:21.793963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:21.794246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:21.794630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] PATH_INFO: `/v3/users/3bf18b67ae18480998d64c65b930d909` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:21.795813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf18b67ae18480998d64c65b930d909)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:21.797873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:21.857653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf18b67ae18480998d64c65b930d909'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:21.866109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf18b67ae18480998d64c65b930d909'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:21.866902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf18b67ae18480998d64c65b930d909'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:21.867593 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf18b67ae18480998d64c65b930d909'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:21.880803 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf18b67ae18480998d64c65b930d909'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:21.897455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2428] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:21 2022] DELETE /identity/v3/users/3bf18b67ae18480998d64c65b930d909 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:21.900164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:21.900164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:21.900880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:21.936282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:21.937481 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:22.023054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1220/2429] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:22.034082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.036579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:22.113469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162df302-71be-45ca-906f-e87ffe1d7b7e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.118367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.119046 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.119581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.119866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.129714 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.131945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5799f-8f44-4030-9574-ef5193c4a28b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.141896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2430] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:22.477739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.479419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:22.493605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33e609-8215-40f4-9ac8-5ef0bc3ea2fb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.498402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.499487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.499645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.499984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] PATH_INFO: `/v3/projects/7780e9307802426983f60a198c879c14` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.501331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7780e9307802426983f60a198c879c14)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.503313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.551202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7780e9307802426983f60a198c879c14', 'request_id': 'req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:22.556769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7780e9307802426983f60a198c879c14', 'request_id': 'req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:22.564787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7780e9307802426983f60a198c879c14', 'request_id': 'req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:22.575825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] The token cache is being invalidate because project 7780e9307802426983f60a198c879c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:22.576137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:22.576752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ea63ba5-9e80-41e3-83b7-38f576dcf4e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:22.578288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1221/2431] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:22 2022] DELETE /identity/v3/projects/7780e9307802426983f60a198c879c14 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:22.584384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.584676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.584958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.613700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:22.614942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:22.623801 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:22.681536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2432] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:22.690140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.690988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.692318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.718926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:22.720422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:22.732487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1222/2433] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:22.736308 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.746285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a14244-2318-4ffd-9cf4-0c7c40119955 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.753435 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.754521 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.754778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.755053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.755777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:40:22.756114 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.757746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.772689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2434] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:40:22.777317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.787807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa67fbd9-5991-4189-88c9-02940234392a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.792524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.793244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.793585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.793938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.794722 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.796718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.838192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1223/2435] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:22 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:22.842994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.855675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e11763-c8d6-49c7-a9e3-cdd2fd1c4d72 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.924006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.924847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.925216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.926145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.926979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.929410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.954398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2436] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:22 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:22.959024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:22.970677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163b0c1-7385-4cf0-9599-0ced4bf03039 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:22.976047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:22.976872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:22.977150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:22.977564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:22.978354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:22.980188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:22.994350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1224/2437] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:22.998745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:23.024426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b8407-b396-4a51-a13e-b75e34989ceb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:23.031346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:23.032535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:23.033230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:23.033898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:23.035081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:23.037002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a7f75-eb93-41ab-b553-dda184ca024c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:23.051705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2438] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:23.056575 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:23.068309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d9097-9c31-4830-a0af-85c373e37aee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:23.073318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:23.074423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:23.074702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:23.075022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] PATH_INFO: `/v3/projects/769f37bc94e24c3482bb039c5a6e7291/users/8c940eba35194b8db4af6eaf2df8b135/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:23.092287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769f37bc94e24c3482bb039c5a6e7291, user_id=8c940eba35194b8db4af6eaf2df8b135, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:23.095629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:23.112278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1225/2439] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:23 2022] PUT /identity/v3/projects/769f37bc94e24c3482bb039c5a6e7291/users/8c940eba35194b8db4af6eaf2df8b135/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:23.121725 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2970-9b31-4130-9c75-a7d91a6d69ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:23.122463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2970-9b31-4130-9c75-a7d91a6d69ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:23.123446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a2970-9b31-4130-9c75-a7d91a6d69ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:23.150806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c31a2970-9b31-4130-9c75-a7d91a6d69ee None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:23.152589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c31a2970-9b31-4130-9c75-a7d91a6d69ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:23.230405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2440] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:23.241810 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:23.245530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:23.282597 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:23.334449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1ab9b-219c-436e-8ec3-c9b0b9c04241 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:23.340258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:23.340907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:23.341475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:23.341620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:23.354203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:23.355739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:23.368078 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1226/2441] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:24.260321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc64-16f9-48aa-a080-ac4dac1bda13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:24.261232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc64-16f9-48aa-a080-ac4dac1bda13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:24.261733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3fc64-16f9-48aa-a080-ac4dac1bda13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:24.335506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ce3fc64-16f9-48aa-a080-ac4dac1bda13 None None] MFA Rules not processed for user `8c940eba35194b8db4af6eaf2df8b135`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:24.337071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ce3fc64-16f9-48aa-a080-ac4dac1bda13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:24.368608 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:24.445514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2442] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:24.453522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:24.455234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:24.557702 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcadfc-a4e4-427b-9881-be294816417a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:24.561895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:24.562538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:24.562818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:24.563122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:24.572084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:24.573527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:24.583169 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1227/2443] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:28.165097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c52ecc-44e5-4585-8299-0efe3894fc76 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:28.165097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c52ecc-44e5-4585-8299-0efe3894fc76 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:28.165097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c52ecc-44e5-4585-8299-0efe3894fc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:28.192344 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14c52ecc-44e5-4585-8299-0efe3894fc76 None None] MFA Rules not processed for user `8c940eba35194b8db4af6eaf2df8b135`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:28.193720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14c52ecc-44e5-4585-8299-0efe3894fc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:28.219316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2444] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:32.205814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:32.210728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:32.226589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f347d-9926-492d-8692-9a97697f1ae2 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:32.231609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:32.232850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:32.232850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:32.233140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] PATH_INFO: `/v3/users/6d3e2599cbca4ac78d8d53927b881c1f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:32.236287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3e2599cbca4ac78d8d53927b881c1f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:32.238862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:32.297994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3e2599cbca4ac78d8d53927b881c1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:32.306829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3e2599cbca4ac78d8d53927b881c1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:32.308048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3e2599cbca4ac78d8d53927b881c1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:32.322891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3e2599cbca4ac78d8d53927b881c1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:32.323757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3e2599cbca4ac78d8d53927b881c1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:32.345749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1228/2445] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:32 2022] DELETE /identity/v3/users/6d3e2599cbca4ac78d8d53927b881c1f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:32.351844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:32.352760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:32.352760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:32.373346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:32.374611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:32.456316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2446] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:32.466543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:32.468560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:32.519071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:32.536542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3fafc-dd6b-4237-bd48-351c5af7c1b2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:32.541365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:32.542057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:32.542346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:32.542714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:32.554255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:32.555778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd8fcd-b2f5-4be2-86c9-38426b48e04e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:32.566718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1229/2447] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:32.924518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:32.926439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:32.958303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a5436-a6c1-4237-9e18-a20c041400df None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:32.968564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:32.969594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:32.969862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:32.970134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] PATH_INFO: `/v3/projects/9c1a34b570ee4757a87ae6fad5dc234e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:32.971460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1a34b570ee4757a87ae6fad5dc234e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:32.973663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.033483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1a34b570ee4757a87ae6fad5dc234e', 'request_id': 'req-60546d5b-3dc9-4176-8c50-af41a1836b1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:33.040721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1a34b570ee4757a87ae6fad5dc234e', 'request_id': 'req-60546d5b-3dc9-4176-8c50-af41a1836b1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:33.053007 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1a34b570ee4757a87ae6fad5dc234e', 'request_id': 'req-60546d5b-3dc9-4176-8c50-af41a1836b1d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:33.065316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] The token cache is being invalidate because project 9c1a34b570ee4757a87ae6fad5dc234e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:33.065640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:33.066277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-60546d5b-3dc9-4176-8c50-af41a1836b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:33.068293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2448] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:32 2022] DELETE /identity/v3/projects/9c1a34b570ee4757a87ae6fad5dc234e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:33.073148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.073554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.073828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.096331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:33.098056 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:33.186285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1230/2449] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:33.187671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.187966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.188255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.208017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:33.209386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:33.217483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2450] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:33.222450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.234653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f6723-b538-4eb5-8021-5ef20849890d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.240803 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.242134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.242625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.243000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.243868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:40:33.244370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.246167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.259089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1231/2451] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:40:33.263328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.272713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c76beb-033a-478e-80af-cfac5acd6dff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.277886 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.278546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.278827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.279103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.279792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.281755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.303741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:33.313591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2452] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:33 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:33.317804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.328905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a07b8f0-6c95-4702-a986-d36e19eb54f9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.393363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.394080 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.394356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.394712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.395452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.397693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.423933 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1232/2453] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:33 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:40:33.425210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.436081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b903c2-4a8f-429b-bd99-b9ecc55c35d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.441133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.441862 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.442177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.442636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.443402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.445059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.455896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2454] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:33.461196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.478864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b966478-52a6-4dda-a303-79ee82852e02 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.485887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.486837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.487217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.487760 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.488536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.490852 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00810c4f-85c5-4297-b7f8-ace51a1cb593 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.507085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1233/2455] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:40:33.511324 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.522005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ce545-e626-4e12-aa19-8cee41beb13d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.527203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.528164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.528443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.528717 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] PATH_INFO: `/v3/projects/44bf8abe9e5d45a4b4719abaaa704e62/users/922afe9fd2c44c0899949cfe8e85e38e/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.545333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44bf8abe9e5d45a4b4719abaaa704e62, user_id=922afe9fd2c44c0899949cfe8e85e38e, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.548241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.563861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2456] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:33 2022] PUT /identity/v3/projects/44bf8abe9e5d45a4b4719abaaa704e62/users/922afe9fd2c44c0899949cfe8e85e38e/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:33.574754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaaf4b-77b0-4c67-a529-4b0cbf09e653 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.575237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaaf4b-77b0-4c67-a529-4b0cbf09e653 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.575512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaaf4b-77b0-4c67-a529-4b0cbf09e653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.604179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dfaaaf4b-77b0-4c67-a529-4b0cbf09e653 None None] MFA Rules not processed for user `922afe9fd2c44c0899949cfe8e85e38e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:33.605395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dfaaaf4b-77b0-4c67-a529-4b0cbf09e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:33.642022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:33.693435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1234/2457] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:33.704747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:33.707053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:33.775443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036129bb-d4da-443b-b014-d5292e8c8e65 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:33.779320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:33.779909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:33.780249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:33.780524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:33.789439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:33.790902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:33.800297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2458] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:39.888179 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fec42a-9d1d-4685-af62-2299daa60b29 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:39.888807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fec42a-9d1d-4685-af62-2299daa60b29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:39.888807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fec42a-9d1d-4685-af62-2299daa60b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:39.933024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b5fec42a-9d1d-4685-af62-2299daa60b29 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:39.934663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5fec42a-9d1d-4685-af62-2299daa60b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:40.008297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1235/2459] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:40:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:40.026917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:40.029485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:40.128848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f329b993-2b99-4554-bb83-b566b5d8019a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:40.138126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:40.138795 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:40.139077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:40.139354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:40.152971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:40.154778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:40.170600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2460] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:40:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:40:44.690298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:44.690903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:44.690903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:44.711690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:44.713683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:44.722048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1236/2461] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:40:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:44.740314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:44.751401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d1530-825d-4b3f-9c24-555efaaf2166 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:44.757665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:44.758688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:44.759216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:44.759695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:44.770566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:44.772490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:44.787215 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2462] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:40:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:40:49.601369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:49.602972 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:49.693149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a6cba-72d1-444c-84ef-9882a7422b4e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:49.710413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:49.711411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:49.711589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:49.711863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] PATH_INFO: `/v3/users/0f0285314f124d228986e7f638833d04` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:49.713561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0285314f124d228986e7f638833d04)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:49.715694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:49.813006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0285314f124d228986e7f638833d04'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:49.822445 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0285314f124d228986e7f638833d04'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:49.823152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0285314f124d228986e7f638833d04'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:49.829671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:49.833480 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0285314f124d228986e7f638833d04'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:49.834496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-148ae34d-178a-4dc7-8c7e-b0b51fb5783a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0285314f124d228986e7f638833d04'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:49.871708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1237/2463] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:49 2022] DELETE /identity/v3/users/0f0285314f124d228986e7f638833d04 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:50.111173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.130357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446529a-1ec6-4e5c-aeef-feaffdfc168e service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.211607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:50.233612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.234536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.234974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.235415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] PATH_INFO: `/v3/projects/80a3c5535f9040e58e055a7277514da1` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.236771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a3c5535f9040e58e055a7277514da1)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.239264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.366748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a3c5535f9040e58e055a7277514da1', 'request_id': 'req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:50.372454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a3c5535f9040e58e055a7277514da1', 'request_id': 'req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:50.391524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a3c5535f9040e58e055a7277514da1', 'request_id': 'req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:50.406236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] The token cache is being invalidate because project 80a3c5535f9040e58e055a7277514da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:50.406500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:50.407128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-38dc4ce4-0a6e-4895-9d6a-ac5887f93345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:50.412223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2464] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:50 2022] DELETE /identity/v3/projects/80a3c5535f9040e58e055a7277514da1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:50.414312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.414634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.414885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.444382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:50.445609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:50.608077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1238/2465] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:50.613848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.613848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.613848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.636102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:50.637814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:50.649357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2466] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:40:50.655417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.667576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea329be3-7283-4007-b55f-ac7c16870fa9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.676194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.677918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.678266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.678594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.679453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:40:50.680084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.681846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.710910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1239/2467] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:40:50.711726 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.726138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc7a7-8081-4a2f-864a-5dd8219ed5e8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.731384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.732166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.732509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.732843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.733594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.735537 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.788097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2468] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:50 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:50.792905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.803581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f695183-1b51-45d9-853f-7c40b300a5d6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.870441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.871164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.871411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.871660 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.872373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.874291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.895185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1240/2469] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:50 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:50.899727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.910297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2d1fd-7843-4725-9ab3-04d928d27fa4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.915587 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.916241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.916518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.916788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.917505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.919120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.930272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2470] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:50.934530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.944828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78243df4-93dc-4ecf-b815-18b4607fd427 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.949946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.950612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.950890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.951159 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:50.951885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:50.953559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016d63-8942-4213-9a31-44efb084f1ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:50.964313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1241/2471] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:50.968384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:50.979172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb033de7-8454-4f45-ac14-06394e8573ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:50.983786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:50.984751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:50.985033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:50.985305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] PATH_INFO: `/v3/projects/dc2810000275432fb1337151b6883e90/users/0a3a64d88e89402b907789c7f030e080/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:51.000427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc2810000275432fb1337151b6883e90, user_id=0a3a64d88e89402b907789c7f030e080, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:51.003108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:51.019582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2472] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:50 2022] PUT /identity/v3/projects/dc2810000275432fb1337151b6883e90/users/0a3a64d88e89402b907789c7f030e080/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:51.025154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:51.025574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:51.026799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:51.049601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:51.051578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:51.106860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:51.130468 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1242/2473] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:51.139825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:51.141535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:51.203088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b3c6b-004b-488e-8e11-fdecf3ff0a02 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:51.206763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:51.207620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:51.207882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:51.208206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:51.217666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:51.219229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942031e9-c9d2-4cd4-b7bf-c4bdfce1349f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:51.229288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2474] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:56.388248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:56.389728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:56.403292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2ca41-7397-43ca-8891-01a1aa9aca02 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:56.409888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:56.411167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:56.411626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:56.412151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] PATH_INFO: `/v3/users/922afe9fd2c44c0899949cfe8e85e38e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:56.414076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] RBAC: Authorizing `identity:delete_user(user_id=922afe9fd2c44c0899949cfe8e85e38e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:56.416291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:56.494855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922afe9fd2c44c0899949cfe8e85e38e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:56.523957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922afe9fd2c44c0899949cfe8e85e38e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:56.526799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922afe9fd2c44c0899949cfe8e85e38e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:56.537177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922afe9fd2c44c0899949cfe8e85e38e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:56.539246 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922afe9fd2c44c0899949cfe8e85e38e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:56.583736 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1243/2475] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:40:56 2022] DELETE /identity/v3/users/922afe9fd2c44c0899949cfe8e85e38e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:56.588360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecdfec-d97e-474d-ae6b-762be7b414c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:56.588871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecdfec-d97e-474d-ae6b-762be7b414c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:56.588966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecdfec-d97e-474d-ae6b-762be7b414c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:56.631926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-70ecdfec-d97e-474d-ae6b-762be7b414c9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:56.634195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-70ecdfec-d97e-474d-ae6b-762be7b414c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:56.646125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:56.773460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2476] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:56.784681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:56.786388 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:56.849387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0b134-dbb2-453f-b9eb-0ffaf3e7360e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:56.853967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:56.854789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:56.855228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:56.855790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:56.865938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:56.867418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:56.884826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1244/2477] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:56.925015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:56.925664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:56.928991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:56.970664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] MFA Rules not processed for user `0a3a64d88e89402b907789c7f030e080`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:56.971990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:57.092797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2478] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:40:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:57.095019 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.096349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:57.117028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c195d-84fa-48cf-b545-adf37da74dbf service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.122501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.123370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.123646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.123912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.124656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.129701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.189202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1245/2479] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:40:57 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:57.192980 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.213742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c82f2d-6700-4897-8651-f72a73d6a5d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.309387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.311764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:57.344202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.346230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.346509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.346807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.348004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.354097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.447974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2480] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:40:57 2022] POST /identity/v3/users => generated 295 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:40:57.452528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.482961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2532bea4-f867-42f8-912c-3444c0bedbe9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.489016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:57.497515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.500573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.500799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.501078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.505522 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.507721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.542011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2481] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:57.543790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.553212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64e2a-7640-4896-a525-fb9836580211 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.563469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b3d02-08c6-48e6-933f-83e0d32f1984 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.564763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.567227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.567510 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.567788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] PATH_INFO: `/v3/projects/44bf8abe9e5d45a4b4719abaaa704e62` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.569553 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44bf8abe9e5d45a4b4719abaaa704e62)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.572766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.579378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.580456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.580737 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.581011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] PATH_INFO: `/v3/projects/c5765604521448658675300905017afe/users/09088c96d0b24f109c801a8e4fd1dbbf/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.625955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5765604521448658675300905017afe, user_id=09088c96d0b24f109c801a8e4fd1dbbf, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.630964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.643042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:57.669871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2482] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:57 2022] PUT /identity/v3/projects/c5765604521448658675300905017afe/users/09088c96d0b24f109c801a8e4fd1dbbf/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:57.678249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.710778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44bf8abe9e5d45a4b4719abaaa704e62', 'request_id': 'req-3da62b60-712b-4d16-aee7-902f39e0b321'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:57.716673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44bf8abe9e5d45a4b4719abaaa704e62', 'request_id': 'req-3da62b60-712b-4d16-aee7-902f39e0b321'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:57.729018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44bf8abe9e5d45a4b4719abaaa704e62', 'request_id': 'req-3da62b60-712b-4d16-aee7-902f39e0b321'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:57.760096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] The token cache is being invalidate because project 44bf8abe9e5d45a4b4719abaaa704e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:40:57.760467 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:57.761211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:40:57.763966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1246/2483] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:40:57 2022] DELETE /identity/v3/projects/44bf8abe9e5d45a4b4719abaaa704e62 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:57.795112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469a0f8-8c01-4d95-8b99-f64e5b80d2db admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:57.931147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:57.932069 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:57.933727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:57.934097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:57.935949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:57.942926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5951b6e-09a3-472b-af54-8bca3a67e9ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:57.965938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2484] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:40:57.974315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:57.980059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:58.016889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62b60-712b-4d16-aee7-902f39e0b321 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:58.029243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:58.030285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:58.030573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:58.030849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] PATH_INFO: `/v3/projects/c5765604521448658675300905017afe/users/09088c96d0b24f109c801a8e4fd1dbbf/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:58.035598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5765604521448658675300905017afe, user_id=09088c96d0b24f109c801a8e4fd1dbbf, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:58.038494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed819bc6-1398-4815-a150-c9bff083da5b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:58.058741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1247/2485] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:40:57 2022] PUT /identity/v3/projects/c5765604521448658675300905017afe/users/09088c96d0b24f109c801a8e4fd1dbbf/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:40:58.902744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:58.902744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:58.903441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:58.905731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2486] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:40:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:40:58.913737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028196e-2013-4ba9-8266-b1720c792754 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:58.914937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028196e-2013-4ba9-8266-b1720c792754 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:58.914937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028196e-2013-4ba9-8266-b1720c792754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:58.994312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c028196e-2013-4ba9-8266-b1720c792754 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:58.995843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c028196e-2013-4ba9-8266-b1720c792754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:59.130264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1248/2487] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:40:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:59.148257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:59.150432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:59.293331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5fe474-b523-4b89-9473-357769ab7a9a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:59.298042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:59.298942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.299642 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.300075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.315219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:59.317064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06a18e-5847-4322-a800-65164d0eee99 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:59.333549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2488] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:59.421402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.422262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.422262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.423581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1249/2489] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:40:59.431879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a177-c9d2-4ed7-9cdb-fd0266494709 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.432220 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a177-c9d2-4ed7-9cdb-fd0266494709 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.432502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4a177-c9d2-4ed7-9cdb-fd0266494709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.454852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1ea4a177-c9d2-4ed7-9cdb-fd0266494709 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:59.457518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ea4a177-c9d2-4ed7-9cdb-fd0266494709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:59.478756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:59.537583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2490] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:40:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:59.556824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:59.558891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:59.576754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b37e3-eb14-427e-b886-5d95b345d109 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:59.582868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:59.583741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.584203 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.584555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.612522 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:59.615225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aac6b4-7369-4fac-bfe4-ff51a32609b7 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:59.621296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:59.639410 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1250/2491] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:40:59.740815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76fa74-3218-407c-b100-2be26bdcb408 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.741210 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76fa74-3218-407c-b100-2be26bdcb408 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.741407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76fa74-3218-407c-b100-2be26bdcb408 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.743050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2492] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:40:59.754307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.757266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.758084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.786924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:40:59.788196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:59.800236 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1251/2493] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:40:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:40:59.812841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76fa74-3218-407c-b100-2be26bdcb408 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:59.843026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76fa74-3218-407c-b100-2be26bdcb408 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:59.848824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:59.849595 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:59.849979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:59.850353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:59.873291 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:59.874865 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99add4d-64c5-491f-8456-f7d7bec5f53e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:59.890518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2494] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:40:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:00.282687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:00.285804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:00.470039 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ecedb-882f-45e2-8b31-56b9c8957788 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:00.477366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:00.478899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:00.479178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:00.479449 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] PATH_INFO: `/v3/users/5f2c10ac03fd41e7a595ee63e4a4789b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:00.482396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f2c10ac03fd41e7a595ee63e4a4789b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:00.484427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:00.584560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f2c10ac03fd41e7a595ee63e4a4789b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:00.594630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f2c10ac03fd41e7a595ee63e4a4789b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:00.595328 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f2c10ac03fd41e7a595ee63e4a4789b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:00.604983 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f2c10ac03fd41e7a595ee63e4a4789b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:00.605809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f2c10ac03fd41e7a595ee63e4a4789b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:00.652507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1252/2495] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:41:00 2022] DELETE /identity/v3/users/5f2c10ac03fd41e7a595ee63e4a4789b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:41:00.654293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:00.654780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:00.654859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:00.707451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:00.711787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:00.859532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2496] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:00.871400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:00.874305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:01.031302 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17466bef-c6c7-4038-8690-231fdaf693fd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:01.071288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:01.074625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:01.075088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:01.076454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:01.115644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:01.117228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5550-ab94-4f62-be3a-1ab8683cee67 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:01.152251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1253/2497] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:01.669251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:01.670909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:01.733528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb907425-4a73-4758-877b-f1b25991bcb2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:01.768763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:01.781452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:01.781750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:01.782036 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] PATH_INFO: `/v3/projects/934272ce955f46d489f97a6a4179791a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:01.793307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=934272ce955f46d489f97a6a4179791a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:01.798073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:01.894645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934272ce955f46d489f97a6a4179791a', 'request_id': 'req-4a977fb0-903f-4730-9354-64fe42159d56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:01.901178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934272ce955f46d489f97a6a4179791a', 'request_id': 'req-4a977fb0-903f-4730-9354-64fe42159d56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:01.913391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934272ce955f46d489f97a6a4179791a', 'request_id': 'req-4a977fb0-903f-4730-9354-64fe42159d56'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:01.958940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] The token cache is being invalidate because project 934272ce955f46d489f97a6a4179791a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:41:01.959233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:01.959971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a977fb0-903f-4730-9354-64fe42159d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:01.963016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2498] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:41:01 2022] DELETE /identity/v3/projects/934272ce955f46d489f97a6a4179791a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:41:01.969641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:01.970262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:01.970262 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:01.996954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:01.998513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:02.042942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:02.162977 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1254/2499] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:02.166750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.167335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.167842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.200806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:02.202451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:02.217070 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2500] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:02.221603 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.239711 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842dd1e9-1a4d-4a74-a227-47e2543408fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.253758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.254799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.255204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.255572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.257324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:41:02.266154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.268225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.318822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1255/2501] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:41:02.324885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.337401 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650a437-e9a9-4147-a4bd-526e52832d7d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.349615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.351284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.351776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.352274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.353242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.359965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.421025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2502] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:41:02 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:41:02.422965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.443308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe24116-57cb-4eb5-9599-644d76f9ff6c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.594873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.595567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.595849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.598157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.598998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.601243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.655282 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1256/2503] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:41:02 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:41:02.659831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.682144 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce673e-8cb0-4e8f-974a-bbd4585f1f99 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.696433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.697293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.697764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.698173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.700300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.705659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.734421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.738505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2504] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:41:02.747348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4f64-818f-417b-99c8-101d99ac1de3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.775837 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.776602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.776878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.777148 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.777882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.779853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8734b6-79d1-4326-a373-a5f168ba14e3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.834872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1257/2505] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:41:02.837218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:02.846333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:02.864190 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e4467-c6dd-40c5-bc02-c9ca89bf7c4d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:02.883665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:02.884905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.885301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.886303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] PATH_INFO: `/v3/projects/25ca54924ba846d2ad7a1972f1cb351c/users/f7e65818d4f8416c9bc1b5491d76c8ec/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:02.921926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ca54924ba846d2ad7a1972f1cb351c, user_id=f7e65818d4f8416c9bc1b5491d76c8ec, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:02.927588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:02.964673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2506] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:41:02 2022] PUT /identity/v3/projects/25ca54924ba846d2ad7a1972f1cb351c/users/f7e65818d4f8416c9bc1b5491d76c8ec/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:41:02.975655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbe8f-aa14-433a-884f-8062d5667173 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:02.976177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbe8f-aa14-433a-884f-8062d5667173 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:02.976286 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717fbe8f-aa14-433a-884f-8062d5667173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:03.032363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-717fbe8f-aa14-433a-884f-8062d5667173 None None] MFA Rules not processed for user `f7e65818d4f8416c9bc1b5491d76c8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:03.034084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-717fbe8f-aa14-433a-884f-8062d5667173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:03.170136 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1258/2507] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:03.180992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:03.182890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:03.314536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f18c37-e247-40c6-9577-75e40d00167e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:03.335785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:03.336604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:03.336981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:03.337927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:03.349745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:03.351817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d69c12-6085-41ca-83f7-f34e24675488 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:03.367840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2508] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:04.947375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:04.947942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:04.947942 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:04.972734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] MFA Rules not processed for user `f7e65818d4f8416c9bc1b5491d76c8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:04.974146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:04.981277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1259/2509] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:05.006380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fe1b-2180-4403-84be-dd4093e05568 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:05.006942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fe1b-2180-4403-84be-dd4093e05568 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:05.006942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fe1b-2180-4403-84be-dd4093e05568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:05.070480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fde5fe1b-2180-4403-84be-dd4093e05568 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:05.071742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fde5fe1b-2180-4403-84be-dd4093e05568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:05.142028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2510] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:05.168101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:05.169756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:05.175420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:05.204002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:05.204321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:05.204621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:05.235245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] MFA Rules not processed for user `09088c96d0b24f109c801a8e4fd1dbbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:05.236831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:05.258255 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:05.284944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c385-7687-433c-a774-186a1e8236dd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:05.292418 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:05.293450 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:05.293883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:05.294337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:05.304556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:05.306723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:05.312169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2511] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:41:05.323032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1260/2512] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:05.325035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:05.326594 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:05.439051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74b813-5794-4eac-9fcd-7e3f313508f7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:05.450788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:05.451811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:05.452274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:05.452706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:05.457249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:05.556922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:05.558579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:05.586248 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2513] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:15.319321 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=39 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:15.564568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=75 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:16.489735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:16.489735 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:16.527632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:16.529297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:16.598091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7ed7-880b-4115-9079-13de2340d10b service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:16.599303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beadf1d-0762-496d-b182-a66ad22b3924 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:16.603245 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:16.603889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:16.604196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:16.604347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:16.604469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:16.604999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:16.605278 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:16.605553 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:16.606589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:16.607430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:16.618356 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:16.619131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:16.619934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdba98-1b0e-4f94-9d55-c529bc866d07 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:16.620769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3eb3-90ac-4f6b-9958-69080b6eb53d service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:16.630831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1261/2514] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:16.631364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2515] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:41:26.626250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:26.626892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=32 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:35.833435 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:35.834010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:35.834010 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:35.851667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] MFA Rules not processed for user `0a3a64d88e89402b907789c7f030e080`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:35.852890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:35.859886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1262/2516] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:40.087064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11fee0-343f-45c0-ac03-21c770c1e2c2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:40.087574 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11fee0-343f-45c0-ac03-21c770c1e2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:40.087693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11fee0-343f-45c0-ac03-21c770c1e2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:40.108208 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3d11fee0-343f-45c0-ac03-21c770c1e2c2 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:40.109433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d11fee0-343f-45c0-ac03-21c770c1e2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:40.117779 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2517] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:41:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:40.139363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:40.155507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eb00c-03b3-449b-8b13-c277cc358fc3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:40.161946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:40.162608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:40.162881 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:40.163155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:40.171868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:40.173500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:40.188309 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1263/2518] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:42.094704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14355f7a-499c-4cfe-a354-9f6a87c3b10d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:42.095192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14355f7a-499c-4cfe-a354-9f6a87c3b10d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:42.095299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14355f7a-499c-4cfe-a354-9f6a87c3b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:42.114868 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14355f7a-499c-4cfe-a354-9f6a87c3b10d None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:42.116107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14355f7a-499c-4cfe-a354-9f6a87c3b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:42.123599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2519] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:42.135461 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:42.146183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5795938-08dd-44e4-8a16-768a9c810f48 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:42.151626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:42.152313 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:42.152595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:42.152867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:42.162067 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:42.163682 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:42.174348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1264/2520] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:42.826317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67441cec-5235-46bf-a2e7-a91fce0dcbb0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:42.826606 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67441cec-5235-46bf-a2e7-a91fce0dcbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:42.826882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67441cec-5235-46bf-a2e7-a91fce0dcbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:42.846017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-67441cec-5235-46bf-a2e7-a91fce0dcbb0 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:42.847523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-67441cec-5235-46bf-a2e7-a91fce0dcbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:42.858128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2521] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:41:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:42.871645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:42.883144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7971912-7d56-48ef-9faa-a87a3572826d service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:42.889523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:42.890375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:42.890662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:42.891551 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:42.921259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:42.923611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:42.933400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1265/2522] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:43.159160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64dce2-6831-4e88-bb92-80c3e58e59c1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:43.159686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64dce2-6831-4e88-bb92-80c3e58e59c1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:43.159686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64dce2-6831-4e88-bb92-80c3e58e59c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:43.179842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0a64dce2-6831-4e88-bb92-80c3e58e59c1 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:43.181093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a64dce2-6831-4e88-bb92-80c3e58e59c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:43.188965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2523] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:43.201139 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:43.213635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff534-bd9a-40f6-921f-a0f82f317ca6 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:43.219944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:43.220835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:43.221118 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:43.221395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:43.230248 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:43.231846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:43.240753 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1266/2524] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:43.571107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:43.571396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:43.571668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:43.594415 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:43.595624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:43.603922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2525] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:43.616336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:43.626591 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f24fb-bb99-444d-a395-7913372d9dc7 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:43.632155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:43.632799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:43.633076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:43.633346 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:43.641787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:43.643377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bf947-056b-48e0-a96b-e1afe7812b92 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:43.652174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1267/2526] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:41:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:41:45.880547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:45.945571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a6c17-4687-4a29-9498-135ca3eaa7e7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:45.951236 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:45.952193 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:45.952477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:45.952749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] PATH_INFO: `/v3/users/f7e65818d4f8416c9bc1b5491d76c8ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:45.953981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e65818d4f8416c9bc1b5491d76c8ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:45.955939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:46.006824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e65818d4f8416c9bc1b5491d76c8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.014707 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e65818d4f8416c9bc1b5491d76c8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.015464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e65818d4f8416c9bc1b5491d76c8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.016073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e65818d4f8416c9bc1b5491d76c8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.028733 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e65818d4f8416c9bc1b5491d76c8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.041461 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2527] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:41:45 2022] DELETE /identity/v3/users/f7e65818d4f8416c9bc1b5491d76c8ec => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:41:46.048300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:46.048866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:46.048866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:46.070393 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:46.071659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:46.145573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1268/2528] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:46.156348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:46.158766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:46.225555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:46.232240 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25ed4a-347b-4dd1-89a3-8a3c7bbce88f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:46.237016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:46.237816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:46.238271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:46.238551 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:46.248040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:46.249511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e3afb-2473-4d3b-945d-61cbc1753334 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:46.259462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2529] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:46.502828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:46.504743 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:46.517637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd006f-610e-4963-b27c-ea80bb03c896 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:46.529814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:46.530689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:46.531050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:46.531237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] PATH_INFO: `/v3/projects/25ca54924ba846d2ad7a1972f1cb351c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:46.532524 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ca54924ba846d2ad7a1972f1cb351c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:46.534423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:46.587205 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ca54924ba846d2ad7a1972f1cb351c', 'request_id': 'req-596e4fd1-5152-4d84-9b7c-843f2bf6e469'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.593003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ca54924ba846d2ad7a1972f1cb351c', 'request_id': 'req-596e4fd1-5152-4d84-9b7c-843f2bf6e469'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.600892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ca54924ba846d2ad7a1972f1cb351c', 'request_id': 'req-596e4fd1-5152-4d84-9b7c-843f2bf6e469'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.612088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] The token cache is being invalidate because project 25ca54924ba846d2ad7a1972f1cb351c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:41:46.612378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.613626 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:41:46.615183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1269/2530] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:41:46 2022] DELETE /identity/v3/projects/25ca54924ba846d2ad7a1972f1cb351c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:41:56.255244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:56.609412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:56.651789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354afbc-ec57-47dd-940e-f298690bcbb0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:56.652052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354afbc-ec57-47dd-940e-f298690bcbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:56.652372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354afbc-ec57-47dd-940e-f298690bcbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:56.671140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0354afbc-ec57-47dd-940e-f298690bcbb0 None None] MFA Rules not processed for user `8c940eba35194b8db4af6eaf2df8b135`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:41:56.672399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0354afbc-ec57-47dd-940e-f298690bcbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:56.728510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2531] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:41:58.353770 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:58.356108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:58.421566 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e4fd1-5152-4d84-9b7c-843f2bf6e469 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:58.425487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:58.426657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:58.426657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:58.426957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:58.429962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:41:58.503707 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:58.505396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71323bac-74fd-44aa-9e46-4d2bd0e8d9b0 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:58.514952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1270/2532] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:41:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:06.721613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:08.510907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=50 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:22.778433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:22.779149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:22.779268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:22.797752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] MFA Rules not processed for user `6d07884e92bc4cfe95d2fd99c40f25f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:42:22.798971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:22.854905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2533] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:42:32.847659 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:37.921555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:37.922455 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:37.922821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:37.941037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] MFA Rules not processed for user `9bb20bf4c31d44ac9dd9ef0db9a2a63a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:42:37.942250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:38.012327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1271/2534] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:42:48.005794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=25 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:59.388174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:59.389713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:59.467369 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608f095-9a16-41a8-baaa-ab9a0996078f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:59.473633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:59.474531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:59.474810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:59.475079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] PATH_INFO: `/v3/users/8c940eba35194b8db4af6eaf2df8b135` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:59.476445 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c940eba35194b8db4af6eaf2df8b135)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:59.478420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:59.527602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c940eba35194b8db4af6eaf2df8b135'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.536258 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c940eba35194b8db4af6eaf2df8b135'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.536856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c940eba35194b8db4af6eaf2df8b135'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.537585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c940eba35194b8db4af6eaf2df8b135'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.551600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-754570ab-bba4-4cc9-b7b9-7026dcc7604e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c940eba35194b8db4af6eaf2df8b135'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.566381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2535] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:42:59 2022] DELETE /identity/v3/users/8c940eba35194b8db4af6eaf2df8b135 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:42:59.752593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:59.764538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34bfdb-4a10-4d90-87b7-190b01280478 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:59.827310 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:59.828272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:59.828527 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:59.828857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] PATH_INFO: `/v3/projects/769f37bc94e24c3482bb039c5a6e7291` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:59.830087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769f37bc94e24c3482bb039c5a6e7291)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:59.831997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:59.881885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769f37bc94e24c3482bb039c5a6e7291', 'request_id': 'req-3be1fb6c-e147-4246-9e1a-0343e55a4d56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.887602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769f37bc94e24c3482bb039c5a6e7291', 'request_id': 'req-3be1fb6c-e147-4246-9e1a-0343e55a4d56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.896784 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769f37bc94e24c3482bb039c5a6e7291', 'request_id': 'req-3be1fb6c-e147-4246-9e1a-0343e55a4d56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.908210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] The token cache is being invalidate because project 769f37bc94e24c3482bb039c5a6e7291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:42:59.908490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.909156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3be1fb6c-e147-4246-9e1a-0343e55a4d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:42:59.910739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1272/2536] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:42:59 2022] DELETE /identity/v3/projects/769f37bc94e24c3482bb039c5a6e7291 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:42:59.915890 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:59.916235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:59.916556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:59.936160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:42:59.937366 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:59.998687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2537] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:00.006633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.006937 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.007243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.026426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:00.027674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:00.036124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1273/2538] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:00.040561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.050359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167517f-d269-4bdc-a474-76f779ac6c50 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.055174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.055826 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.056128 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.056399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.057117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:43:00.057419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.059029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.070591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2539] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:43:00.074969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.085374 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e983de8-1bcc-4c25-973a-c8e623be938b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.091545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.092230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.092506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.092776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.093440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.095478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.122768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1274/2540] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:43:00 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:43:00.127086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.137583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22dc92-6176-4ded-b677-9f5239dea04d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.141631 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:00.199227 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.199903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.200206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.200485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.201170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.203119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.223662 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2541] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:43:00 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:43:00.227884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.238865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b33e1-033a-4c8f-ac70-f03b756dc0ec admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.243891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.244586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.244866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.245140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.245864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.247490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.258255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1275/2542] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:43:00.262109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.273147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79c4c6-3bc0-4231-9783-57310b9a3742 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.278199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.278891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.279176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.279436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.280189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.281899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10339968-314d-49a9-a8b2-c429e60859f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.292506 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2543] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:43:00.296606 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.308007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b3798-336c-4bcc-aaa9-645b68177dee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.314363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.315202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.315479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.315747 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] PATH_INFO: `/v3/projects/b05a5e216dd141dabe29dfc71abee8ee/users/423be519855441a6863badc453828bea/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.331489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b05a5e216dd141dabe29dfc71abee8ee, user_id=423be519855441a6863badc453828bea, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.334183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.349126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1276/2544] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:43:00 2022] PUT /identity/v3/projects/b05a5e216dd141dabe29dfc71abee8ee/users/423be519855441a6863badc453828bea/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:43:00.354478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.354766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.355042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.374359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:00.375602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:00.437471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2545] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:00.447219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:00.449081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:00.482157 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:00.511919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994bb20-4e6d-4e44-b665-95eb6002e1ee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:00.515734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:00.516512 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:00.516791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:00.517066 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:00.526855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:00.528343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d56e24-2e48-4f38-abdd-d925eb15713e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:00.537713 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1277/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:43:03.881385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:03.883584 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:03.949106 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983fab9-838e-4c60-b8d1-094a55374ea5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:03.953230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:03.953512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:03.953794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:03.954099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:03.960181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:04.010267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:04.063341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:04.065804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:04.079333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2547] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:04.553856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:04.554352 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:04.554447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:04.581751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] MFA Rules not processed for user `423be519855441a6863badc453828bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:04.582982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:04.651244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1278/2548] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:04.655906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:04.671319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76c3f8-b50c-4d32-a2f4-8309a7090980 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:04.676810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:04.677656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:04.678088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:04.678371 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:04.679102 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:04.681173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:04.720063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2549] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:43:04 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:43:04.721976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:04.733879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d6445-a326-4605-8b54-c2ca8c4ebd46 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:04.793362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:04.794090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:04.794369 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:04.794642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:04.795348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:04.797443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:04.823621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1279/2550] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:43:04 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:43:04.824912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:04.836914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52f107-2822-4d7c-bc83-f310eaf9c7db admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:04.857052 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:04.858137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:04.859097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:04.859857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:04.861011 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:04.862840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:04.882432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2551] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:43:04.883037 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:04.894571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97557173-321e-4c92-b9f0-6fe74e20f04c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:04.899482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:04.903146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:04.903146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:04.903146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] PATH_INFO: `/v3/projects/f70c5efe3285406f84ab092456a78e9c/users/835a08fb07624518b7143abadbad57da/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:04.925147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c5efe3285406f84ab092456a78e9c, user_id=835a08fb07624518b7143abadbad57da, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:04.927879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:04.945985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1280/2552] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:43:04 2022] PUT /identity/v3/projects/f70c5efe3285406f84ab092456a78e9c/users/835a08fb07624518b7143abadbad57da/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:43:04.950284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:04.961377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092cbf5-93e7-4229-9fe4-2f3da16cd0a2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:05.034860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:05.035552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:05.035830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:05.036141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:05.036899 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:05.038542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:05.049297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2553] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:43:05.053325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:05.064472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2723d18-d996-4633-bac2-de632e6b3571 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:05.069720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:05.070644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:05.070899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:05.071194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] PATH_INFO: `/v3/projects/f70c5efe3285406f84ab092456a78e9c/users/835a08fb07624518b7143abadbad57da/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:05.073227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70c5efe3285406f84ab092456a78e9c, user_id=835a08fb07624518b7143abadbad57da, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:05.075840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2d042-5c1a-4e3e-9b42-6cbb08b0020d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:05.088698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1281/2554] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:43:05 2022] PUT /identity/v3/projects/f70c5efe3285406f84ab092456a78e9c/users/835a08fb07624518b7143abadbad57da/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:43:06.930953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:07.024647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac795aa-a9d7-4ef5-a541-ae88d8df1038 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:07.082050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:07.088263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:07.089297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:07.089785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:07.090063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:07.092618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:07.181825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:07.183325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:07.195492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2555] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:10.065710 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008827-5136-475d-9251-faf16c36a765 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:10.066235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008827-5136-475d-9251-faf16c36a765 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:10.066235 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07008827-5136-475d-9251-faf16c36a765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:10.102069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-07008827-5136-475d-9251-faf16c36a765 None None] MFA Rules not processed for user `835a08fb07624518b7143abadbad57da`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:10.103307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07008827-5136-475d-9251-faf16c36a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:10.108832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:10.240134 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1282/2556] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:10.253322 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:10.254880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:10.343145 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13283ec-680c-4123-bc15-a81dbd252e33 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:10.348777 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:10.351188 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:10.351785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:10.352766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:10.446878 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:10.448442 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:10.465277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2557] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:10.625773 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:10.627469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:10.627469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:10.647146 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:10.648258 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:10.723271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1283/2558] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:10.783702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:10.785112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:10.856524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:10.890132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524a047-571e-4560-90da-35c36d5ca37f service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:10.896796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:10.897559 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:10.897907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:10.898243 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:10.907490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:10.909277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ab14b-b61c-46d7-9e42-08ae430e53f6 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:10.918486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2559] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:43:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:43:14.418537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:14.536249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12841ee8-2366-4c86-b0b0-289a72eb7dcd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:14.543432 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:14.544769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:14.545127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:14.545475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:14.549116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:14.657020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:14.662995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:14.665890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:14.677222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1284/2560] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:20.363665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:20.364242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:20.364242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:20.382920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] MFA Rules not processed for user `423be519855441a6863badc453828bea`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:20.384157 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:20.391497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2561] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:20.669230 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:20.679583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10742d00-6995-4fd2-8220-c8e76a806e80 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:20.684149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:20.684756 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:20.685031 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:20.685298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:20.687486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:20.698100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:20.699610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20264711-804a-4eed-b63d-c26119e88b82 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:20.708184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1285/2562] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:21.041705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:21.043480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:21.115958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdbac9-505c-4291-8807-5ff335828f15 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:21.120818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:21.121467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:21.121814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:21.122018 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:21.123769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:21.135376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:21.137015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:21.156232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2563] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:23.950005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:23.950543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:23.950761 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:23.973904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:23.975438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:23.984303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1286/2564] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:43:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:43:24.011983 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:24.029062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8811f-8d27-40e9-b422-42af12a0f85b service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:24.034604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:24.035337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:24.035630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:24.035901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:24.044963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:24.046906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:24.056403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2565] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:43:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 15:43:28.936166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:28.937732 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:28.955677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3eee1-ec71-44b8-9886-8ecea99389e2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:28.963502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:28.964218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:28.964496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:28.964768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:28.966781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:29.038594 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:29.040268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:29.050459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1287/2566] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:33.209831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:33.211563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:33.225034 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5d50a-df3b-44f5-94db-e77d56e2bbd9 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:33.228785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:33.229438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:33.229721 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:33.229994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:33.231804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:33.241416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:33.242866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:33.251078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2567] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:33.794077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:33.804271 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a388-93f9-4954-80be-76dd1c0c96ce service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:33.808814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:33.809704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:33.809986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:33.810324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:33.812407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:33.888530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:33.890312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:33.902641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1288/2568] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:38.131448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.133003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:38.148690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cf7a83-f7ca-4c72-966f-176465ed64e6 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.153863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.154586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.154949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.155447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.157319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:38.173161 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.174945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:38.199217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2569] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:38.206191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.208249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:38.222170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0642ec-da43-466b-96a2-506fc53c137b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.228560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.230044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.230334 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.230721 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] PATH_INFO: `/v3/users/6d07884e92bc4cfe95d2fd99c40f25f0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.231839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d07884e92bc4cfe95d2fd99c40f25f0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.234011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:38.285069 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d07884e92bc4cfe95d2fd99c40f25f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.293050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d07884e92bc4cfe95d2fd99c40f25f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.293776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d07884e92bc4cfe95d2fd99c40f25f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.300895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d07884e92bc4cfe95d2fd99c40f25f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.301521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-44603546-cd68-4955-adb9-65144ef8e3e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d07884e92bc4cfe95d2fd99c40f25f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.302826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:38.323484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1289/2570] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:43:38 2022] DELETE /identity/v3/users/6d07884e92bc4cfe95d2fd99c40f25f0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:43:38.513265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.525917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628075-4b6a-437c-9ebf-4a88c6ad9b57 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.590798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.591961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.592276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.592550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] PATH_INFO: `/v3/projects/e280d545f6064b5c91289a77e2491511` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.593767 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e280d545f6064b5c91289a77e2491511)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.595723 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:38.613050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:38.644505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e280d545f6064b5c91289a77e2491511', 'request_id': 'req-88062bf7-a851-4f2a-99be-9ae05b2101d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.650275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e280d545f6064b5c91289a77e2491511', 'request_id': 'req-88062bf7-a851-4f2a-99be-9ae05b2101d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.658387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e280d545f6064b5c91289a77e2491511', 'request_id': 'req-88062bf7-a851-4f2a-99be-9ae05b2101d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.670015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] The token cache is being invalidate because project e280d545f6064b5c91289a77e2491511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:43:38.670301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.670937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88062bf7-a851-4f2a-99be-9ae05b2101d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:43:38.672563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2571] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:43:38 2022] DELETE /identity/v3/projects/e280d545f6064b5c91289a77e2491511 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:43:38.677834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.678125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.678399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.699520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:38.700763 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:38.764771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1290/2572] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:38.774199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.774581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.774966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.798252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:38.799456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:38.808469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2573] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:38.812832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.823144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f6f50-cb6d-4f6d-a22a-bc06c435b87c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.829561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.830213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.830490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.830759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.831478 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:43:38.831781 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.833426 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:38.846625 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1291/2574] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:43:38.852632 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.862353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bf422-e2a3-47a6-8a7d-725ab774ee64 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.869827 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.870485 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.870763 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.871033 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.871719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.873746 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:38.904623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2575] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:43:38 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:43:38.908573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:38.919253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cd288-4476-4ea9-a18c-6bf6a7e93b44 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:38.982026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:38.982538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:38.982976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:38.983362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:38.984084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:38.986021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:39.015012 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1292/2576] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:43:38 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:43:39.016317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:39.027373 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a728-b416-4276-ad94-ed59d06d836d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:39.032202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:39.032887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:39.033196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:39.033470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:39.034338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:39.036017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:39.047421 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2577] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:43:39.051475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:39.062386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19509f1-b349-4bb8-ae5a-5aa0efae72b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:39.067100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:39.067762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:39.068184 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:39.068456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:39.069189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:39.070795 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cdb56-b130-4730-8e93-ca2e86670518 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:39.081525 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1293/2578] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:43:39.085806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:39.096104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73ff82-bc21-4217-bd61-255e1b406ea7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:39.100804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:39.101708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:39.101990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:39.102262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] PATH_INFO: `/v3/projects/f92c7d44a8de4b2ab85ab081276300fd/users/ad8f617cb0b14532b3ad514a0430e54f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:39.118646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f92c7d44a8de4b2ab85ab081276300fd, user_id=ad8f617cb0b14532b3ad514a0430e54f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:39.121665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:39.136547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2579] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:43:39 2022] PUT /identity/v3/projects/f92c7d44a8de4b2ab85ab081276300fd/users/ad8f617cb0b14532b3ad514a0430e54f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:43:39.141573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93832-0eb0-4668-b147-4dc7c5617670 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:39.141931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93832-0eb0-4668-b147-4dc7c5617670 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:39.142311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93832-0eb0-4668-b147-4dc7c5617670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:39.160589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-02d93832-0eb0-4668-b147-4dc7c5617670 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:39.161934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-02d93832-0eb0-4668-b147-4dc7c5617670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:39.203140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:39.222766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1294/2580] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:39.232985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:39.234771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:39.296059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97159d87-ef48-48e1-b096-50609756b66f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:39.300399 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:39.301053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:39.301342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:39.301620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:39.311354 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:39.312850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:39.322792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2581] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:43.998737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cef7-4a27-4003-a9c6-7211b8fe48d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:43.999215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cef7-4a27-4003-a9c6-7211b8fe48d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:43.999343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929cef7-4a27-4003-a9c6-7211b8fe48d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:44.032544 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0929cef7-4a27-4003-a9c6-7211b8fe48d2 None None] MFA Rules not processed for user `ad8f617cb0b14532b3ad514a0430e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:44.033785 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0929cef7-4a27-4003-a9c6-7211b8fe48d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:44.105569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1295/2582] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:44.120778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:44.122602 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:44.186177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:44.225058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7165859-248a-486f-a3db-f449d78deaa3 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:44.231017 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:44.231814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:44.232129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:44.232407 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:44.243022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:44.244901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:44.279084 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2583] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:45.951446 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:45.952007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:45.952007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:45.972606 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] MFA Rules not processed for user `ad8f617cb0b14532b3ad514a0430e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:45.973824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:45.990359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1296/2584] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:43:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:43:46.391988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:46.402136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b9334-873d-4a0c-816e-3b5aebd62517 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:46.406784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:46.407818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:46.408132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:46.408396 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:46.410552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:46.421398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:46.422943 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37586954-d5a8-4387-80cc-fdfd062d2a03 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:46.431961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2585] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:49.400457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:49.415093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f26c20-ad7b-46ab-8222-89f20a490a01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:49.421928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:49.422668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:49.422948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:49.423251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:49.425133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:49.447048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:49.448880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:49.460835 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1297/2586] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:51.561440 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:51.561957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:51.562051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:51.580365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:43:51.581560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:51.642356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2587] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:43:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:43:51.654959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:51.656545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:51.728371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66588a9-c5ba-4511-87b5-55cdf9f42703 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:51.733621 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:51.734270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:51.734552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:51.734823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:51.744226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:51.745866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:51.757003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1298/2588] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:43:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:43:53.844019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:53.845566 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:53.861524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e5af-fb71-4d10-8d9a-27337e34ff56 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:53.865989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:53.866940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:53.867464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:53.867988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:53.870176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:53.979945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:53.981757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:53.995020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2589] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:54.352319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:54.376768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efc9f11-1224-4dd8-8c2b-632e59ca3688 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:54.389834 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:54.390899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:54.391476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:54.392403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:54.394622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:54.410379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:54.412596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b3405-46a3-49c7-b42d-aeaba7ce5759 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:54.454477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1299/2590] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:56.601667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:56.603879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:56.635256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f2a2f-8326-4583-a7e7-a53b01ae1164 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:56.668829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:56.669525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:56.669994 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:56.670281 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:56.679133 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:56.704335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:56.705985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661795c2-0fc4-4995-8edd-c7d7e6def61e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:56.726129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2591] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:04.421909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:06.719118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:06.944371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:06.944924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:06.944924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:06.946547 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1300/2592] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:44:06.953901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d502f-6596-4713-9501-6a0794dedbb8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:06.954186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d502f-6596-4713-9501-6a0794dedbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:06.954488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d502f-6596-4713-9501-6a0794dedbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:06.974127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f01d502f-6596-4713-9501-6a0794dedbb8 None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:06.975329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f01d502f-6596-4713-9501-6a0794dedbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:06.985647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2593] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:44:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:06.997846 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:06.999673 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:07.073400 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81221a8-e7d4-40f9-9ed8-48c375cb6c84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:07.077047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ae9d48a9a9384ada870d31cdbc9c90e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:07.077692 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.077965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.078234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.091434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:07.092587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a8e6d-3211-4351-a190-81bd5132a3bc service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:07.103811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1301/2594] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:44:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:44:07.174306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851806-52f0-49e3-838a-7af893175027 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.174591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851806-52f0-49e3-838a-7af893175027 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.174935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851806-52f0-49e3-838a-7af893175027 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.176592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2595] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:44:07.181907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28206278-e006-42ec-bbf0-195a2ab70a89 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.182192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28206278-e006-42ec-bbf0-195a2ab70a89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.182500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28206278-e006-42ec-bbf0-195a2ab70a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.201396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-28206278-e006-42ec-bbf0-195a2ab70a89 None None] MFA Rules not processed for user `fbe062ad83f3490282626add7af4947a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:07.202632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28206278-e006-42ec-bbf0-195a2ab70a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:07.210095 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1302/2596] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:44:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:44:07.222126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851806-52f0-49e3-838a-7af893175027 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:07.234679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851806-52f0-49e3-838a-7af893175027 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:07.238641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:07.239279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.239555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.239825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.247256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:07.248738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976a08c-cc36-46da-945f-0ad17212e7c4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:07.256740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2597] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:44:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:07.340304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcde9-b9de-4d81-b97f-594c74767f44 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.340587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcde9-b9de-4d81-b97f-594c74767f44 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.340853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabcde9-b9de-4d81-b97f-594c74767f44 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.342214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1303/2598] 10.222.0.55 () {54 vars in 921 bytes} [Tue May 24 15:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:44:07.347843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc787d-fa7c-4a64-bb49-084f6f983bbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.348169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc787d-fa7c-4a64-bb49-084f6f983bbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.348443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc787d-fa7c-4a64-bb49-084f6f983bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.367071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6abc787d-fa7c-4a64-bb49-084f6f983bbb None None] MFA Rules not processed for user `c87b5039c28044988e7d8c418c286994`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:07.368388 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6abc787d-fa7c-4a64-bb49-084f6f983bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:07.378534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2599] 10.222.0.55 () {58 vars in 1018 bytes} [Tue May 24 15:44:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:07.392569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabcde9-b9de-4d81-b97f-594c74767f44 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:07.408117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabcde9-b9de-4d81-b97f-594c74767f44 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:07.412216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:07.412892 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:07.413176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:07.413452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:07.425001 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:07.426456 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:07.438076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1304/2600] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:44:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:11.117785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246543-bb6d-4bab-ac13-a98060b30789 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:11.118313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246543-bb6d-4bab-ac13-a98060b30789 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:11.118313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246543-bb6d-4bab-ac13-a98060b30789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:11.139712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4246543-bb6d-4bab-ac13-a98060b30789 None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:11.140974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4246543-bb6d-4bab-ac13-a98060b30789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:11.153427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2601] 10.222.0.55 () {58 vars in 1009 bytes} [Tue May 24 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:11.176058 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:11.188728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e854-2c7b-4c9c-bab4-3a3df4d3ae44 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:11.194742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:11.195399 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:11.195679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:11.195955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:11.205469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:11.207098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843435be-a5c0-4b34-b636-aee55640d2ff service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:11.216746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1305/2602] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:44:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:44:21.133006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=15 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:21.212938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:22.947260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:22.947893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:22.947893 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:22.966298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] MFA Rules not processed for user `0a3a64d88e89402b907789c7f030e080`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:22.967532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:23.026598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2603] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:44:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:28.360492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:28.361003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:28.361287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:28.379296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:28.380536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:28.387932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1306/2604] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:44:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:28.400811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:28.413299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c501e-9030-4a2e-8bed-10c4f22820ce None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:28.419719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:28.420404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:28.420689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:28.420962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:28.430124 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:28.431734 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:28.440076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2605] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 15:44:38.373861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:38.436667 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=31 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:10.154635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:10.164995 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76c98a-7050-41d9-86d5-c90d0946dadb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:10.169576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:10.170215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:10.170507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:10.170764 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:10.172819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:10.185619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:10.187055 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:10.198087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1307/2606] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:45:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:45:13.358562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:13.358562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:13.436560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:13.438924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:13.451956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c31fb-4fb5-4b45-96a7-e9e30b99f827 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:13.455789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa1a018-70a7-4d6c-92ff-f56a9f16fb8b service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:13.457752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:13.458636 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:13.458913 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:13.459357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] PATH_INFO: `/v3/users/9bb20bf4c31d44ac9dd9ef0db9a2a63a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:13.460569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb20bf4c31d44ac9dd9ef0db9a2a63a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:13.460997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:13.461647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:13.461926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:13.462196 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:13.462520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:13.463925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:13.476091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:13.477699 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:13.490098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2607] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:45:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:45:13.518953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb20bf4c31d44ac9dd9ef0db9a2a63a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.527329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb20bf4c31d44ac9dd9ef0db9a2a63a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.527935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb20bf4c31d44ac9dd9ef0db9a2a63a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.535598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb20bf4c31d44ac9dd9ef0db9a2a63a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.536402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b43b0d0d-abdc-4b75-85e9-bbb4e74954bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb20bf4c31d44ac9dd9ef0db9a2a63a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.555268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1308/2608] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:45:13 2022] DELETE /identity/v3/users/9bb20bf4c31d44ac9dd9ef0db9a2a63a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:13.738653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:13.757589 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b2ef8-1f33-46dc-b4b7-7cb9a32101d8 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:13.834242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:13.835918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:13.836319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:13.836666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] PATH_INFO: `/v3/projects/fd0613f4cbb3461a982738c683c3ab03` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:13.838851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0613f4cbb3461a982738c683c3ab03)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:13.841656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:13.919297 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0613f4cbb3461a982738c683c3ab03', 'request_id': 'req-73450207-ca76-4649-a489-971aa08ff1af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.935964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0613f4cbb3461a982738c683c3ab03', 'request_id': 'req-73450207-ca76-4649-a489-971aa08ff1af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.949524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0613f4cbb3461a982738c683c3ab03', 'request_id': 'req-73450207-ca76-4649-a489-971aa08ff1af'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.967633 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] The token cache is being invalidate because project fd0613f4cbb3461a982738c683c3ab03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:45:13.968331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.969693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-73450207-ca76-4649-a489-971aa08ff1af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:13.971491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2609] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:45:13 2022] DELETE /identity/v3/projects/fd0613f4cbb3461a982738c683c3ab03 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:13.979771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:13.980427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:13.980914 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.002505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:14.005929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:14.122234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1309/2610] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:45:14.130425 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.131076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.131906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.156134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:14.157555 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:14.172963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2611] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:45:14.174403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.192108 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10630a21-71e4-4fa0-8bbe-77b78e05e069 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.201466 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.202229 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.202565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.202902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.203847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:45:14.204614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.206324 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.226022 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.226189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1310/2612] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:45:14.240567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db70ac7-2399-4821-a123-ae49db9d8e4a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.254239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.255672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.256383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.256990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.257931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.260665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.274771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:14.321572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.327892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2613] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:45:14 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:45:14.337535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f1718b-9d3c-4573-a4b8-9ad9b7f2956b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.398903 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:14.417694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.418501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.418630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.418907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.419945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.421598 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.445068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1311/2614] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:45:14 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:45:14.449209 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.462362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a408ca-891f-4c03-b853-90c374710c48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.467572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.468300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.468583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.468857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.469590 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.471347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.488372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2615] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:45:14.489638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.502765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c9b54-2c73-469e-a472-bb2aa747b913 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.508505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.509385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.509502 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.509726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.510465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.512210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320274b7-2b7e-47c8-9618-abe30b5a10f0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.526845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1312/2616] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:45:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:45:14.528532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.539749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5f114-6c09-444d-9fe1-1ef540f590d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.544657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.545570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.545834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.546105 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] PATH_INFO: `/v3/projects/a30edd891b96456fbba6409a815f5fd8/users/0f8105d1de0d45218576d74a2ba4abe4/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.568455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a30edd891b96456fbba6409a815f5fd8, user_id=0f8105d1de0d45218576d74a2ba4abe4, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.571312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.589958 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2617] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:45:14 2022] PUT /identity/v3/projects/a30edd891b96456fbba6409a815f5fd8/users/0f8105d1de0d45218576d74a2ba4abe4/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:14.595460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd2eed-fec6-47c1-8aab-a810dde05b70 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.595758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd2eed-fec6-47c1-8aab-a810dde05b70 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.596064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd2eed-fec6-47c1-8aab-a810dde05b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.615900 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-43bd2eed-fec6-47c1-8aab-a810dde05b70 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:14.617318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43bd2eed-fec6-47c1-8aab-a810dde05b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:14.691254 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1313/2618] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:45:14.697810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:14.699678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:14.758567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc4f45-1dfc-4393-bd86-f639741aa52a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:14.762310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:14.762967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:14.763246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:14.763517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:14.773447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:14.774894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:14.784771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2619] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:45:19.013694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4518d6-1aeb-4bd2-9428-103724ef853c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:19.014556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4518d6-1aeb-4bd2-9428-103724ef853c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:19.014556 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4518d6-1aeb-4bd2-9428-103724ef853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:19.041337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c4518d6-1aeb-4bd2-9428-103724ef853c None None] MFA Rules not processed for user `0f8105d1de0d45218576d74a2ba4abe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:19.042487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c4518d6-1aeb-4bd2-9428-103724ef853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:19.112366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1314/2620] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:45:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:45:19.127815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:19.129433 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:19.214897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5b28f-053d-4af8-bfdf-9a960a44cdef service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:19.220755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:19.221742 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:19.222482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:19.222835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:19.236812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:19.238776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8a2a5-535a-4e23-8e55-16a3dccb9c88 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:19.253338 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2621] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:45:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:45:20.814379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:20.814830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:20.815005 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:20.838042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] MFA Rules not processed for user `0f8105d1de0d45218576d74a2ba4abe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:20.839877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:20.847968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1315/2622] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:45:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:45:21.756147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:21.756663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:21.756776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:21.780756 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] MFA Rules not processed for user `69a0a21790ed496ba0e1e397a9daea09`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:45:21.782053 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:21.820572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:21.852296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2623] 10.222.0.55 () {58 vars in 967 bytes} [Tue May 24 15:45:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:45:21.877716 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:21.879560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:21.934499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:21.956922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17d9d2-cbc6-4d21-92f6-a34e18653543 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:21.962171 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb122c4f54840a98cd9fc932fb2856c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:21.962873 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:21.963153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:21.963421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:21.973121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:21.974714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:21.985071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1316/2624] 10.222.0.55 () {60 vars in 1384 bytes} [Tue May 24 15:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:45:29.808082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:29.810997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:29.828067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0dce6-2792-4868-b659-39534dc94ffd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:29.831852 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:29.832793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:29.833071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:29.833340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] PATH_INFO: `/v3/users/423be519855441a6863badc453828bea` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:29.834577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] RBAC: Authorizing `identity:delete_user(user_id=423be519855441a6863badc453828bea)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:29.836672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:29.888130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423be519855441a6863badc453828bea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:29.897807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423be519855441a6863badc453828bea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:29.898505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423be519855441a6863badc453828bea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:29.899392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423be519855441a6863badc453828bea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:29.911648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423be519855441a6863badc453828bea'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:29.925772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2625] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:45:29 2022] DELETE /identity/v3/users/423be519855441a6863badc453828bea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:30.118048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:30.132875 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb749b-a024-4831-8686-95663ea6a3f5 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:30.193739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:30.194610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:30.194887 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:30.195161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] PATH_INFO: `/v3/projects/b05a5e216dd141dabe29dfc71abee8ee` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:30.196470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b05a5e216dd141dabe29dfc71abee8ee)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:30.198436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:30.252068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b05a5e216dd141dabe29dfc71abee8ee', 'request_id': 'req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.261415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b05a5e216dd141dabe29dfc71abee8ee', 'request_id': 'req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.271489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b05a5e216dd141dabe29dfc71abee8ee', 'request_id': 'req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.284731 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] The token cache is being invalidate because project b05a5e216dd141dabe29dfc71abee8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:45:30.285017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.285724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.287453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1317/2626] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:45:30 2022] DELETE /identity/v3/projects/b05a5e216dd141dabe29dfc71abee8ee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:30.292246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:30.294093 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:30.308363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb7a7-67b0-461c-a43b-1deded40038a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:30.370613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:30.372191 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:30.372573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:30.372785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] PATH_INFO: `/v3/users/835a08fb07624518b7143abadbad57da` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:30.373996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835a08fb07624518b7143abadbad57da)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:30.375913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:30.435295 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835a08fb07624518b7143abadbad57da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.444457 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835a08fb07624518b7143abadbad57da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.446478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835a08fb07624518b7143abadbad57da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.447321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835a08fb07624518b7143abadbad57da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.449104 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:30.467530 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835a08fb07624518b7143abadbad57da'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.500352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2627] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:45:30 2022] DELETE /identity/v3/users/835a08fb07624518b7143abadbad57da => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:30.689357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:45:30.691307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:45:30.707963 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0876b-3ac7-42f1-977c-1a754a4d8c37 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:45:30.772639 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:45:30.773518 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:45:30.773798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:45:30.774072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] PATH_INFO: `/v3/projects/f70c5efe3285406f84ab092456a78e9c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:45:30.775255 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70c5efe3285406f84ab092456a78e9c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:45:30.777222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:45:30.788687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:30.829635 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70c5efe3285406f84ab092456a78e9c', 'request_id': 'req-13612cae-2b6b-4073-b983-c2c79d7705db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.836402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70c5efe3285406f84ab092456a78e9c', 'request_id': 'req-13612cae-2b6b-4073-b983-c2c79d7705db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.847253 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70c5efe3285406f84ab092456a78e9c', 'request_id': 'req-13612cae-2b6b-4073-b983-c2c79d7705db'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.860196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] The token cache is being invalidate because project f70c5efe3285406f84ab092456a78e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:45:30.860529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.861609 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-13612cae-2b6b-4073-b983-c2c79d7705db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:45:30.863338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1318/2628] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:45:30 2022] DELETE /identity/v3/projects/f70c5efe3285406f84ab092456a78e9c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:45:40.479930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:45:40.856654 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:46:19.247183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:19.248764 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:19.312722 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f353-3e9e-4d1f-9765-a432df5834f3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:19.363006 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:19.363657 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:19.363940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:19.364246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:19.366065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:19.440134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:19.441600 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:19.450807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2629] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:46:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:20.525006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bafc-4e19-93f1-546de5a80708 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:20.525552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bafc-4e19-93f1-546de5a80708 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:20.525552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e36873-bafc-4e19-93f1-546de5a80708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:20.545405 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9e36873-bafc-4e19-93f1-546de5a80708 None None] MFA Rules not processed for user `0f8105d1de0d45218576d74a2ba4abe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:46:20.546733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9e36873-bafc-4e19-93f1-546de5a80708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:20.606218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1319/2630] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:46:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:46:22.502394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:22.504125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:22.579806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbd4d3-0cce-43b6-a7b4-9a6c7fa9c7d2 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:22.585027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:22.585698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:22.586026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:22.586303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:22.588289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:22.601937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:22.603669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937f32d1-2679-461d-a0c4-8f45df35c444 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:22.609554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:46:22.618080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2631] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:46:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:26.334421 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365446e-65aa-40b9-b726-59b60bfb4c7b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:26.334945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365446e-65aa-40b9-b726-59b60bfb4c7b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:26.334945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365446e-65aa-40b9-b726-59b60bfb4c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:26.353905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e365446e-65aa-40b9-b726-59b60bfb4c7b None None] MFA Rules not processed for user `0a3a64d88e89402b907789c7f030e080`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:46:26.355109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e365446e-65aa-40b9-b726-59b60bfb4c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:26.362104 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1320/2632] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:46:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:46:36.348151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=28 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:46:39.203375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:39.203892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:39.203892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:39.221964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] MFA Rules not processed for user `ad8f617cb0b14532b3ad514a0430e54f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:46:39.223228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:39.278204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2633] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:46:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:46:49.271204 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=25 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:47:24.542373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:47:24.542940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:47:24.542940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:47:24.563030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] MFA Rules not processed for user `0f8105d1de0d45218576d74a2ba4abe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:47:24.564257 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:47:24.571703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1321/2634] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:47:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:47:34.557811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:04.019942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.022010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:04.089263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abafd4f-7a97-4940-9a18-86a2bb8a4da1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.094872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.095760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.096010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.096314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] PATH_INFO: `/v3/users/ad8f617cb0b14532b3ad514a0430e54f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.097716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8f617cb0b14532b3ad514a0430e54f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.099708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.156183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8f617cb0b14532b3ad514a0430e54f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.164818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8f617cb0b14532b3ad514a0430e54f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.165460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8f617cb0b14532b3ad514a0430e54f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.166072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8f617cb0b14532b3ad514a0430e54f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.178234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a758b232-b299-462a-b244-1692c87f1ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8f617cb0b14532b3ad514a0430e54f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.191560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2635] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:04 2022] DELETE /identity/v3/users/ad8f617cb0b14532b3ad514a0430e54f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:04.374728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.392893 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b019f0a-b179-4fa2-8fc2-ab6a2df797cb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.451733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.452611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.452880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.453173 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] PATH_INFO: `/v3/projects/f92c7d44a8de4b2ab85ab081276300fd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.454349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f92c7d44a8de4b2ab85ab081276300fd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.456294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.504688 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f92c7d44a8de4b2ab85ab081276300fd', 'request_id': 'req-7c2bf05c-af07-4bbf-91e4-5a223104723d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.510197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f92c7d44a8de4b2ab85ab081276300fd', 'request_id': 'req-7c2bf05c-af07-4bbf-91e4-5a223104723d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.518665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f92c7d44a8de4b2ab85ab081276300fd', 'request_id': 'req-7c2bf05c-af07-4bbf-91e4-5a223104723d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.529668 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] The token cache is being invalidate because project f92c7d44a8de4b2ab85ab081276300fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:04.529948 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.530565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7c2bf05c-af07-4bbf-91e4-5a223104723d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:04.532137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1322/2636] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:04 2022] DELETE /identity/v3/projects/f92c7d44a8de4b2ab85ab081276300fd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:04.537239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.537471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.537760 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.556665 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:04.558419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:04.625253 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2637] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:04.633423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.633708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.633982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.653027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:04.654225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:04.662499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1323/2638] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:04.666809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.676582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d40f3-ebea-461a-8f59-7eecc864393c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.681549 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.682198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.682497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.682790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.683508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:04.683807 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.685469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.697815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2639] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:48:04.702526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.711845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddc4ec-bc16-4075-8d07-73f64c36bdca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.717061 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.717704 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.717958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.718250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.718931 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.721020 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.749726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1324/2640] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:04 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:04.753898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.764430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d766c96-89a1-4999-b1a4-9c4e8d0c0615 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.768470 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:04.828452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.829045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.829317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.829591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.830280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.832229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.855907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2641] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:04 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:04.860154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.871365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61449-108f-435e-a578-501bee25ab70 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.876488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.877137 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.877411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.877662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.878382 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.879965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.890818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1325/2642] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:04.894720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.905327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d713d-d1a2-4ee7-ab71-3e8ef7b038eb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.911271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.911930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.912233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.912498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.913221 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.914817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5efbca-3c84-4e97-a9d1-212e425ec1b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.925177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2643] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:04.929554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:04.939634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bab78d7-e263-4e1b-ad54-22c235b35301 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:04.944537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:04.945412 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.945687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.945956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] PATH_INFO: `/v3/projects/6a1471fc4fa54e2f9b013123308d0607/users/9b5e098139dc497785c17c0c764d8ab2/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:04.961283 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1471fc4fa54e2f9b013123308d0607, user_id=9b5e098139dc497785c17c0c764d8ab2, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:04.963960 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:04.978557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1326/2644] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:04 2022] PUT /identity/v3/projects/6a1471fc4fa54e2f9b013123308d0607/users/9b5e098139dc497785c17c0c764d8ab2/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:04.988863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:04.989179 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:04.989455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:05.017487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] MFA Rules not processed for user `9b5e098139dc497785c17c0c764d8ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:05.018695 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:05.085789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2645] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:05.096922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:05.098648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:05.129158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:05.173154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55bc0f7-c0ee-411a-ae1e-9663bd9152fc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:05.180048 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:05.180819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:05.181093 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:05.181372 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:05.189499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:05.191081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:05.200124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1327/2646] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:05.220303 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:05.221783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:05.278898 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb89b4-cefb-4f26-b150-87eb92a944f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:05.282877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:05.283518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:05.283797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:05.284063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:05.286032 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:05.300374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:05.301835 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:05.312979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2647] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:06.766119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:06.776259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1d79f-ed25-4c14-8448-50e23915419c service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:06.780529 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:06.781164 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:06.781448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:06.781723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:06.783592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:06.798126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:06.799555 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b64a536-b27f-4747-9fe2-962db3c1197c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:06.811465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1328/2648] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:08.072147 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:08.090762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:08.139244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843139-3e05-4c29-abf7-dcaa6bdcc51f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:08.147100 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:08.148412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.148794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.149176 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] PATH_INFO: `/v3/users/9b5e098139dc497785c17c0c764d8ab2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.151143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5e098139dc497785c17c0c764d8ab2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:08.154153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:08.208182 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5e098139dc497785c17c0c764d8ab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.216540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5e098139dc497785c17c0c764d8ab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.217167 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5e098139dc497785c17c0c764d8ab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.217780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5e098139dc497785c17c0c764d8ab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.230111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5e098139dc497785c17c0c764d8ab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.243309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2649] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:08 2022] DELETE /identity/v3/users/9b5e098139dc497785c17c0c764d8ab2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:08.248168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.248697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.248697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.267687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:08.268944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:08.331894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1329/2650] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:08.341748 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:08.343251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:08.411420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0fa06-a84b-4d50-8a0a-9b5c9e9fc266 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:08.416274 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:08.416934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.417218 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.417492 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.428588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:08.430058 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa95ba-f85a-4970-a1c9-5528dd5a4a98 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:08.440185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2651] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:48:08.721974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:08.723519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:08.738009 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c135518e-56c9-4f2a-b040-5a88d9e23fe2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:08.742578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:08.743452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.743733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.744007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] PATH_INFO: `/v3/projects/6a1471fc4fa54e2f9b013123308d0607` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.745585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1471fc4fa54e2f9b013123308d0607)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:08.747499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:08.804003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1471fc4fa54e2f9b013123308d0607', 'request_id': 'req-2f8ce7e5-f8be-45db-ad11-333a07519886'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.810730 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1471fc4fa54e2f9b013123308d0607', 'request_id': 'req-2f8ce7e5-f8be-45db-ad11-333a07519886'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.819758 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1471fc4fa54e2f9b013123308d0607', 'request_id': 'req-2f8ce7e5-f8be-45db-ad11-333a07519886'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.832953 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] The token cache is being invalidate because project 6a1471fc4fa54e2f9b013123308d0607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:08.833243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.833940 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2f8ce7e5-f8be-45db-ad11-333a07519886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:08.835577 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1330/2652] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:08 2022] DELETE /identity/v3/projects/6a1471fc4fa54e2f9b013123308d0607 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:08.840766 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.840945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.841235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.860727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:08.861922 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:08.896711 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:08.924605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2653] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:08.932561 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.932848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.933119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.952637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:08.953839 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:08.963536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1331/2654] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:08.968448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:08.982814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeca833-97f0-4e8f-ba78-f4e7be7e91df None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:08.988823 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:08.989488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:08.989768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:08.990127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:08.990907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:08.991217 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:08.992879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.004837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2655] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:48:09.009429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.019477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368cdd34-3061-47f3-870c-60b8e685fb1d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.024967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.025634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.025911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.026207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.026899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.028879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.042239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:09.057018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1332/2656] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:09.061653 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.072066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff6b180-a9cc-48be-a6a5-f4d3f6593cd9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.128612 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.129245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.129479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.129761 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.130439 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.132404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.153846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2657] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/users => generated 279 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:09.158138 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.168831 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074ee02f-2a53-4a13-a95f-13f002b0d08e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.174944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.175604 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.175879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.176176 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.176895 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.178499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.189659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1333/2658] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:09.193811 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.204335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc591c-ee67-455b-90e2-e4ce5acaeb8b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.210408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.211067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.211345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.211613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.212368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.213981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3608998-2968-4b04-93b5-386e8c60a327 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.224452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2659] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:09.228850 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.239160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23469bbb-b575-4979-9788-da4c1421421b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.244081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.244956 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.245231 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.245486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] PATH_INFO: `/v3/projects/1dc21756ec684f77bc7d6ea5119cd5c7/users/2165dff2293d436196cd4c7b0eafcd12/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.261272 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc21756ec684f77bc7d6ea5119cd5c7, user_id=2165dff2293d436196cd4c7b0eafcd12, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.264315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.279818 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1334/2660] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:09 2022] PUT /identity/v3/projects/1dc21756ec684f77bc7d6ea5119cd5c7/users/2165dff2293d436196cd4c7b0eafcd12/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:09.290687 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.290966 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.291279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.319797 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] MFA Rules not processed for user `2165dff2293d436196cd4c7b0eafcd12`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:09.321042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:09.388977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2661] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:09.393790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.456571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aa300b-6fa5-445e-864a-15af7737d252 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.462476 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.463109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.463392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.463666 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.464422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.466342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.493755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1335/2662] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:09.497926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.509249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f072f-2155-4db3-9c2e-0d0bedcfa120 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.543703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:09.573389 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.574078 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.574356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.574656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.575340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.577486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.598351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2663] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:09.602429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.613186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d103d-6e0a-457a-b7ab-23e53e620002 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.618655 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.619311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.619589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.619860 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.620614 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.622366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.633497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1336/2664] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:09.638030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.649465 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee8a8d-b74c-4333-a25b-cb3681270b71 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.655448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.656380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.656652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.656914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] PATH_INFO: `/v3/projects/7a79a8e5a74a48d18478bdf3490b625a/users/ac047ca469be4c55b54d5bf046aa84d3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.673016 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a79a8e5a74a48d18478bdf3490b625a, user_id=ac047ca469be4c55b54d5bf046aa84d3, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.675712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.690845 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2665] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:09 2022] PUT /identity/v3/projects/7a79a8e5a74a48d18478bdf3490b625a/users/ac047ca469be4c55b54d5bf046aa84d3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:09.694865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.705344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd3b7-18b2-40f4-b256-4e9344598ffe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.762320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.763307 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.763581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.763822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.764583 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.766191 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8ea89-2368-474d-85e0-32ce3d96fe4a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.776234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1337/2666] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:48:09.781932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.792934 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238698-58e4-4296-afcf-68b341a4094d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:09.798134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:09.799045 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.799318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.799570 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] PATH_INFO: `/v3/projects/7a79a8e5a74a48d18478bdf3490b625a/users/ac047ca469be4c55b54d5bf046aa84d3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.801599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a79a8e5a74a48d18478bdf3490b625a, user_id=ac047ca469be4c55b54d5bf046aa84d3, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:09.804262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:09.815750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2667] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:09 2022] PUT /identity/v3/projects/7a79a8e5a74a48d18478bdf3490b625a/users/ac047ca469be4c55b54d5bf046aa84d3/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:09.827453 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:09.827739 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:09.828057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:09.856789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] MFA Rules not processed for user `ac047ca469be4c55b54d5bf046aa84d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:09.858228 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:09.875718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:09.933653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1338/2668] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:09.945351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:09.947372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:10.018572 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c28c4-3f88-4dd5-8874-990059a99ba0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:10.023854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:10.024604 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:10.024880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:10.025151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:10.035099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:10.036641 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:10.046298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2669] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:09 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:10.129674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:10.140943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e2aed-50c2-44ae-9247-bd1d192443c9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:10.144970 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:10.145607 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:10.145891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:10.146163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:10.210799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:10.213050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535477d6-8739-4b87-b60d-cd0ddd239ab6 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:10.227441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1339/2670] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:20.041796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=54,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:20.220217 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=49 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:25.379702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:25.448279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645fb8ec-c166-4423-898e-d89b40f26d5c service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:25.456080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:25.456995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:25.457272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:25.457672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] PATH_INFO: `/v3/users/2165dff2293d436196cd4c7b0eafcd12` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:25.459120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2165dff2293d436196cd4c7b0eafcd12)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:25.461142 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:25.514562 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2165dff2293d436196cd4c7b0eafcd12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:25.524203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2165dff2293d436196cd4c7b0eafcd12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:25.525460 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2165dff2293d436196cd4c7b0eafcd12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:25.526064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2165dff2293d436196cd4c7b0eafcd12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:25.539509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2165dff2293d436196cd4c7b0eafcd12'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:25.553123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2671] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:25 2022] DELETE /identity/v3/users/2165dff2293d436196cd4c7b0eafcd12 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:25.557907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:25.558454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:25.558571 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:25.577154 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:25.578394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:25.651807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1340/2672] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:25.662532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:25.664654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:25.732919 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3539743-ffb5-4d35-aee6-1b4e846fd73d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:25.736658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:25.737510 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:25.737790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:25.738065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:25.749755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:25.752447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:25.772552 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2673] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:26.080601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:26.082441 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:26.095971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c5179-827f-4ac0-81fc-eef8e0423ea2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:26.101611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:26.102676 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:26.103051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:26.103576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] PATH_INFO: `/v3/projects/1dc21756ec684f77bc7d6ea5119cd5c7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:26.105000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc21756ec684f77bc7d6ea5119cd5c7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:26.107034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:26.165086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc21756ec684f77bc7d6ea5119cd5c7', 'request_id': 'req-1fdd13d5-038a-4036-af06-df9c4704dfbd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.170958 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc21756ec684f77bc7d6ea5119cd5c7', 'request_id': 'req-1fdd13d5-038a-4036-af06-df9c4704dfbd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.179845 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc21756ec684f77bc7d6ea5119cd5c7', 'request_id': 'req-1fdd13d5-038a-4036-af06-df9c4704dfbd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.193540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] The token cache is being invalidate because project 1dc21756ec684f77bc7d6ea5119cd5c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:26.193929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.194570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fdd13d5-038a-4036-af06-df9c4704dfbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.196386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1341/2674] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:26 2022] DELETE /identity/v3/projects/1dc21756ec684f77bc7d6ea5119cd5c7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:26.200928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:26.222068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdec0e9-f25e-4d83-8053-d121275b85bf service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:26.252953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:26.305318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:26.306244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:26.306523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:26.306796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] PATH_INFO: `/v3/users/ac047ca469be4c55b54d5bf046aa84d3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:26.308232 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac047ca469be4c55b54d5bf046aa84d3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:26.310127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:26.372143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac047ca469be4c55b54d5bf046aa84d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.380968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac047ca469be4c55b54d5bf046aa84d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.382252 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac047ca469be4c55b54d5bf046aa84d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.382977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac047ca469be4c55b54d5bf046aa84d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.399578 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac047ca469be4c55b54d5bf046aa84d3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.414173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2675] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:26 2022] DELETE /identity/v3/users/ac047ca469be4c55b54d5bf046aa84d3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:26.584407 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c04c6f-3826-4e95-acbc-1501b0b5aa0f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:26.584945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c04c6f-3826-4e95-acbc-1501b0b5aa0f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:26.585044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c04c6f-3826-4e95-acbc-1501b0b5aa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:26.604628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-22c04c6f-3826-4e95-acbc-1501b0b5aa0f None None] MFA Rules not processed for user `0f8105d1de0d45218576d74a2ba4abe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:26.605856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22c04c6f-3826-4e95-acbc-1501b0b5aa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:26.675274 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1342/2676] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:26.727300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:26.728993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:26.805251 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0ca28-23bf-4f0e-90ca-0a58d9d53a25 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:26.810062 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:26.810992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:26.811330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:26.811690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] PATH_INFO: `/v3/projects/7a79a8e5a74a48d18478bdf3490b625a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:26.812949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a79a8e5a74a48d18478bdf3490b625a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:26.815040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:26.858796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:26.863363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a79a8e5a74a48d18478bdf3490b625a', 'request_id': 'req-dc8858f4-159a-4e8d-8157-c37898ce13a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.870518 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a79a8e5a74a48d18478bdf3490b625a', 'request_id': 'req-dc8858f4-159a-4e8d-8157-c37898ce13a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.879242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a79a8e5a74a48d18478bdf3490b625a', 'request_id': 'req-dc8858f4-159a-4e8d-8157-c37898ce13a9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.892475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] The token cache is being invalidate because project 7a79a8e5a74a48d18478bdf3490b625a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:26.892475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.893491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-dc8858f4-159a-4e8d-8157-c37898ce13a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:26.898080 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2677] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:26 2022] DELETE /identity/v3/projects/7a79a8e5a74a48d18478bdf3490b625a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:26.899924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:26.900305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:26.901293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:26.919959 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:26.921697 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:26.949694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:26.996564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1343/2678] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:27.004315 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.004683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.005029 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.006723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.008212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.030526 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:27.031879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.043630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2679] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:27.050214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.062841 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1b683-ae6b-425b-a957-509f92e74264 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.070769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.071509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.072012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.072249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.073536 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:27.073937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.074724 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b4d70-b4f8-43ad-a30c-8531e18fe2ed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.075514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.080888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.081559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.081879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.082183 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.085380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.088683 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2680] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:48:27.093137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.096057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.097592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.102967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a25f8-c0ea-404e-ace2-0ab0f346cdb8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.105811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1344/2681] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:48:27.107614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.108296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.108576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.108844 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.109520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.111490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.143955 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2682] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:27.145396 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.157090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a065f-8164-476f-b67e-9ed11565334d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.225671 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.226448 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.226808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.227144 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.228158 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.230147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.252326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1345/2683] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:27.257524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.267127 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93110196-6489-469e-8fd7-ba00284780b9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.272386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.273057 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.273337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.273605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.274332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.275942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.286508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2684] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:27.290742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.301239 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cc2c8-5f5f-4e18-9688-5f52e9cb5ca7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.306534 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.306783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.307102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.307353 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.308140 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.309967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149a472-4b52-4955-9941-628d44471667 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.320866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1346/2685] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:27.325118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.335293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3607608-392c-4306-957f-3a16c0c63f40 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.340515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.341420 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.341701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.341995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] PATH_INFO: `/v3/projects/a16075c869b040418073b79eefb39221/users/6b9bf6181e1e418297afc6346cfbc71d/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.357967 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16075c869b040418073b79eefb39221, user_id=6b9bf6181e1e418297afc6346cfbc71d, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.360883 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.375357 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2686] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:27 2022] PUT /identity/v3/projects/a16075c869b040418073b79eefb39221/users/6b9bf6181e1e418297afc6346cfbc71d/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:27.385486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.385775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.386044 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.413681 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] MFA Rules not processed for user `6b9bf6181e1e418297afc6346cfbc71d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:27.415197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.458490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:27.488335 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1347/2687] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:27.492724 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.571688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8bd760-e7f7-4197-be43-cba2efe16441 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.576981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.577688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.577965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.578242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.578962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.581359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.614413 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.617430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.621953 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2688] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:27.625913 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.675257 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:27.700822 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b264a47-b11f-4eff-b8c4-0413fa360a5f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.704637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9cfe5-916a-47e7-a440-14bc8c4fddd2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.705814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.706516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.706798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.707225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.707930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.709930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.738912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2689] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:27.740015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.750870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea412695-6422-44e4-9b78-428fdd026393 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.755691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.756567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.756847 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.757119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.757882 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.759505 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.774245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2690] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:48:27.775796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.780545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.781297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.781646 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.781989 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.784279 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:27.787245 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947ae28-12a0-46e4-a722-e8b351d96eac admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.791869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.793085 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.793798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.794859 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] PATH_INFO: `/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca/users/805d2ef70b52477aa65c146c15d0a062/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.798155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.799817 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.811106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1348/2691] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:48:27.816064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9802bc0c67949a58fe9e6f218ba9dca, user_id=805d2ef70b52477aa65c146c15d0a062, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.819031 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.833979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2692] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:27 2022] PUT /identity/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca/users/805d2ef70b52477aa65c146c15d0a062/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:27.837927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.857025 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f9d4-cb62-41d5-869f-f8d91f75713f service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.931623 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.932411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.932689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.932965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.933708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.935638 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfae183-e178-47f5-87ae-d6da6730e858 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.946971 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1349/2693] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:48:27.951178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:27.962931 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d252304-a06e-4bcf-b486-48236f8cd3af admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:27.968489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:27.969529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:27.969877 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:27.970214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] PATH_INFO: `/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca/users/805d2ef70b52477aa65c146c15d0a062/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:27.972744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9802bc0c67949a58fe9e6f218ba9dca, user_id=805d2ef70b52477aa65c146c15d0a062, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:27.975828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:27.988911 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2694] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:27 2022] PUT /identity/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca/users/805d2ef70b52477aa65c146c15d0a062/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:28.000928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:28.001216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:28.001489 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:28.031471 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:28.043370 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] MFA Rules not processed for user `805d2ef70b52477aa65c146c15d0a062`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:28.044699 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:28.139793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1350/2695] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:28.151262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:28.153458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:28.227273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f4bd8-ad22-4140-9d81-adddcac8fc70 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:28.231520 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:28.232938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:28.233205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:28.233474 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:28.248020 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:28.250490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:28.263101 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2696] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:29.574867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:29.653008 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8596a5-3797-4f1c-aae1-05f4bd48d02e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:29.656859 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:29.657509 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:29.657786 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:29.658124 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:29.659945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:29.767601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:29.769657 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f5f00d-8ff1-4878-8a7b-95a18d1a73a1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:29.781904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1351/2697] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:29.837769 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:29.858073 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2948f0-cfec-452e-9595-bf78a19dc0fb service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:29.863350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:29.864299 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:29.865168 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:29.865453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] PATH_INFO: `/v3/users/6b9bf6181e1e418297afc6346cfbc71d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:29.866656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9bf6181e1e418297afc6346cfbc71d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:29.868765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:29.936426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9bf6181e1e418297afc6346cfbc71d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:29.947800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9bf6181e1e418297afc6346cfbc71d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:29.949681 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9bf6181e1e418297afc6346cfbc71d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:29.950348 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9bf6181e1e418297afc6346cfbc71d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:29.967817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9bf6181e1e418297afc6346cfbc71d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:29.974508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:29.984446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2698] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:29 2022] DELETE /identity/v3/users/6b9bf6181e1e418297afc6346cfbc71d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:29.989962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:29.990249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:29.991499 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:30.012375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:30.013921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:30.069990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:30.095595 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1352/2699] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:30.129664 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:30.131270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:30.219263 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d1375-2bea-454c-95d0-7a65500f1f76 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:30.224096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:30.224885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:30.225163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:30.225438 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:30.235925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:30.237500 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:30.253040 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2700] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:30.893472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:30.895685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:30.911294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28547354-45ca-444a-b992-7958bb0f37c9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:30.916656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:30.917653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:30.917957 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:30.918234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] PATH_INFO: `/v3/projects/a16075c869b040418073b79eefb39221` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:30.919538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16075c869b040418073b79eefb39221)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:30.921539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:30.986550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16075c869b040418073b79eefb39221', 'request_id': 'req-08363c4d-3ad3-4f1e-a6c6-422802d02759'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:30.992611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16075c869b040418073b79eefb39221', 'request_id': 'req-08363c4d-3ad3-4f1e-a6c6-422802d02759'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.001640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16075c869b040418073b79eefb39221', 'request_id': 'req-08363c4d-3ad3-4f1e-a6c6-422802d02759'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.014617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] The token cache is being invalidate because project a16075c869b040418073b79eefb39221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:31.014877 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.015526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.017150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1353/2701] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:30 2022] DELETE /identity/v3/projects/a16075c869b040418073b79eefb39221 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:31.021999 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:31.032129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8be88-82fa-4458-be8f-d798cb96595f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:31.106730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:31.107817 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:31.108171 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:31.108451 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] PATH_INFO: `/v3/users/805d2ef70b52477aa65c146c15d0a062` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:31.109789 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=805d2ef70b52477aa65c146c15d0a062)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:31.112368 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:31.169840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '805d2ef70b52477aa65c146c15d0a062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.182938 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '805d2ef70b52477aa65c146c15d0a062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.183651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '805d2ef70b52477aa65c146c15d0a062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.184422 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '805d2ef70b52477aa65c146c15d0a062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.211813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-26926f97-1918-433c-9ca0-33be976cdbe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '805d2ef70b52477aa65c146c15d0a062'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.228968 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2702] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:31 2022] DELETE /identity/v3/users/805d2ef70b52477aa65c146c15d0a062 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:31.740165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:31.741922 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:31.764712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08363c4d-3ad3-4f1e-a6c6-422802d02759 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:31.840340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:31.841423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:31.841808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:31.842174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] PATH_INFO: `/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:31.843517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9802bc0c67949a58fe9e6f218ba9dca)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:31.845627 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:31.903318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9802bc0c67949a58fe9e6f218ba9dca', 'request_id': 'req-56acfd8b-ce2a-4c75-a98c-66434ad14438'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.910562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9802bc0c67949a58fe9e6f218ba9dca', 'request_id': 'req-56acfd8b-ce2a-4c75-a98c-66434ad14438'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.921315 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9802bc0c67949a58fe9e6f218ba9dca', 'request_id': 'req-56acfd8b-ce2a-4c75-a98c-66434ad14438'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.929574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:31.936098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] The token cache is being invalidate because project f9802bc0c67949a58fe9e6f218ba9dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:31.936371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.937341 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-56acfd8b-ce2a-4c75-a98c-66434ad14438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:31.941978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1354/2703] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:31 2022] DELETE /identity/v3/projects/f9802bc0c67949a58fe9e6f218ba9dca => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:31.947289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:31.947624 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:31.947906 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:31.967363 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:31.968735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:31.985398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:32.045132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2704] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:32.053610 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.054027 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.054501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.076359 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:32.077662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:32.086800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1355/2705] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:32.091809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.102747 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df14c5-5600-4e76-bfc1-51f037d96c6b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.108305 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.109092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.109432 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.109768 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.110560 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:32.110914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.112942 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.126071 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2706] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:48:32.130470 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.140320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c17ef5-bffa-4d4e-ab9c-8eb637d0fce0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.146285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.147209 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.147762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.148160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.148941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.151430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.184383 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1356/2707] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:48:32.185692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.198701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e85c77-ef61-4204-8937-100196cb2752 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.267535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.269212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.269212 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.269663 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.270459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.272772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.301794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2708] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:32.304338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.316240 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b04a-e4f7-4035-b74b-8c95fcd34786 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.324849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.326317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.326715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.327318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.327911 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.330964 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.349266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1357/2709] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:48:32.351686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.365825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ba634-79d5-44a5-a3c0-a8bed0af7d7f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.377794 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.378137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.378599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.379561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.380428 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.382103 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883421e-49ff-47d0-8206-d2531b031efb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.402627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2710] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:32.404233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.423219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb8b6c1-ec31-40ff-ad3b-a8a657bdbaab admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.430752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.432014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.432570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.432853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] PATH_INFO: `/v3/projects/c56bc4df1a1a4da68e2a6896994ea75f/users/221bc7d35d9d4ed492f6ddd34cdb8db0/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.450417 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56bc4df1a1a4da68e2a6896994ea75f, user_id=221bc7d35d9d4ed492f6ddd34cdb8db0, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.453751 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.469928 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1358/2711] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:32 2022] PUT /identity/v3/projects/c56bc4df1a1a4da68e2a6896994ea75f/users/221bc7d35d9d4ed492f6ddd34cdb8db0/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:32.482277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.482701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.483047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.511280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] MFA Rules not processed for user `221bc7d35d9d4ed492f6ddd34cdb8db0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:32.512894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:32.594441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2712] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:32.596212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.672452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337e2bc-d75f-44b7-b06d-732b9a1705e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.678170 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.678926 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.679814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.679814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.680576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.688064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.728613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1359/2713] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:32.730800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.748286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0b138-61f6-4e2e-8127-bec745f96c81 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.764512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:32.857650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.857650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.857650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.857982 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.858798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.861937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.891496 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2714] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:32 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:32.896306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.911759 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c0226-6088-4ac1-b443-42d4703d4e93 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.917129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.917823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.918107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.918381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.919127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.920824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:32.938323 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1360/2715] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:32.943327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:32.955464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4772028-4b97-407f-b545-a2ee8e36ad61 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:32.963586 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:32.964992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:32.965498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:32.965903 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] PATH_INFO: `/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467/users/5d6d3fb48059450a839a82eff1df7d0d/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:32.983698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b12118d5184dd2b4ce7bce8cbb4467, user_id=5d6d3fb48059450a839a82eff1df7d0d, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:32.986503 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.002205 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2716] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:32 2022] PUT /identity/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467/users/5d6d3fb48059450a839a82eff1df7d0d/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:33.006505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:33.017863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27c516-4693-4bed-8419-aa0b6567a356 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:33.075505 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:33.076233 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.076503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.076775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.077508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:33.079119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190ef40-dcf5-4304-99ff-1c76d2a24b0b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.089824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1361/2717] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:33.093785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:33.105397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378995de-e793-4c20-8923-ffed40b57be2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:33.110597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:33.111688 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.112112 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.112447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] PATH_INFO: `/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467/users/5d6d3fb48059450a839a82eff1df7d0d/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.114690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b12118d5184dd2b4ce7bce8cbb4467, user_id=5d6d3fb48059450a839a82eff1df7d0d, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:33.117381 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.131423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2718] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:33 2022] PUT /identity/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467/users/5d6d3fb48059450a839a82eff1df7d0d/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:33.142266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.142742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.142988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.151101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:33.179023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] MFA Rules not processed for user `5d6d3fb48059450a839a82eff1df7d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:33.180769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:33.270112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1362/2719] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:33.281353 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:33.284170 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:33.359599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130c620-127b-4850-8a39-6988030e5bed admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:33.363367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:33.364050 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.364335 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.364605 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.376400 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:33.378390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7aae67-4399-44f9-a474-785e429dcceb service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.390126 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2720] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:33.479303 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:33.581032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e7b45-acb0-4587-9228-80f0d01775d2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:33.586772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:33.587899 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.588214 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.588488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] PATH_INFO: `/v3/users/221bc7d35d9d4ed492f6ddd34cdb8db0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.589765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=221bc7d35d9d4ed492f6ddd34cdb8db0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:33.591982 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.655823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '221bc7d35d9d4ed492f6ddd34cdb8db0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:33.670013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '221bc7d35d9d4ed492f6ddd34cdb8db0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:33.671294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '221bc7d35d9d4ed492f6ddd34cdb8db0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:33.687870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '221bc7d35d9d4ed492f6ddd34cdb8db0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:33.689552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '221bc7d35d9d4ed492f6ddd34cdb8db0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:33.722801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1363/2721] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:33 2022] DELETE /identity/v3/users/221bc7d35d9d4ed492f6ddd34cdb8db0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:33.723739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.724959 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.725508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.745686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:33.747177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:33.841954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2722] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:33.855994 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:33.857563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:33.870804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:33.932569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73386429-9834-492b-8eab-849ec31b81e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:33.936287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:33.936934 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:33.937210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:33.937484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:33.948376 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:33.950017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:33.968862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1364/2723] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:48:34.720739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:34.722463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:34.734601 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce258c-d003-4f78-8f97-2af40857bf57 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:34.740646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:34.741516 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:34.741793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:34.742063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] PATH_INFO: `/v3/projects/c56bc4df1a1a4da68e2a6896994ea75f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:34.743355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56bc4df1a1a4da68e2a6896994ea75f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:34.745313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:34.748285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:34.809799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56bc4df1a1a4da68e2a6896994ea75f', 'request_id': 'req-4b7b3586-ec9a-49d3-94b6-909aeaace59f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.815441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56bc4df1a1a4da68e2a6896994ea75f', 'request_id': 'req-4b7b3586-ec9a-49d3-94b6-909aeaace59f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.823972 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56bc4df1a1a4da68e2a6896994ea75f', 'request_id': 'req-4b7b3586-ec9a-49d3-94b6-909aeaace59f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.836378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] The token cache is being invalidate because project c56bc4df1a1a4da68e2a6896994ea75f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:34.836698 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.837406 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.839081 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2724] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:34 2022] DELETE /identity/v3/projects/c56bc4df1a1a4da68e2a6896994ea75f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:34.843366 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:34.853500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c2b86-54c7-47c8-a98a-dd676390e9e2 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:34.911288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:34.912278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:34.912585 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:34.912871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] PATH_INFO: `/v3/users/5d6d3fb48059450a839a82eff1df7d0d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:34.914166 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6d3fb48059450a839a82eff1df7d0d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:34.916075 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:34.966488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6d3fb48059450a839a82eff1df7d0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.974161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6d3fb48059450a839a82eff1df7d0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.974857 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6d3fb48059450a839a82eff1df7d0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.982126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6d3fb48059450a839a82eff1df7d0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:34.982762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-26b48554-e6be-45d1-813d-7a7e8853cce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6d3fb48059450a839a82eff1df7d0d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.000689 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1365/2725] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:34 2022] DELETE /identity/v3/users/5d6d3fb48059450a839a82eff1df7d0d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:35.385259 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:35.388068 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:35.404427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b3586-ec9a-49d3-94b6-909aeaace59f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:35.489783 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:35.490548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.490800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.491374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] PATH_INFO: `/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.492325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b12118d5184dd2b4ce7bce8cbb4467)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:35.494238 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:35.546342 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b12118d5184dd2b4ce7bce8cbb4467', 'request_id': 'req-07c12d2e-94ac-4eba-9db7-9aa4977693a8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.552519 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b12118d5184dd2b4ce7bce8cbb4467', 'request_id': 'req-07c12d2e-94ac-4eba-9db7-9aa4977693a8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.562861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b12118d5184dd2b4ce7bce8cbb4467', 'request_id': 'req-07c12d2e-94ac-4eba-9db7-9aa4977693a8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.583998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] The token cache is being invalidate because project 85b12118d5184dd2b4ce7bce8cbb4467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:35.584561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.585408 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c12d2e-94ac-4eba-9db7-9aa4977693a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:35.591270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2726] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:35 2022] DELETE /identity/v3/projects/85b12118d5184dd2b4ce7bce8cbb4467 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:35.592554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.593244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.593703 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.615777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:35.617059 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:35.648427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:35.709754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1366/2727] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:35.720941 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62954067-fd77-4eda-b745-34186c764522 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.721483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62954067-fd77-4eda-b745-34186c764522 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.722108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62954067-fd77-4eda-b745-34186c764522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.757912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-62954067-fd77-4eda-b745-34186c764522 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:35.761491 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62954067-fd77-4eda-b745-34186c764522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:35.772113 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2728] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:35.776978 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:35.789304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50e251-e4f9-4ef1-9bad-7772ebc3365d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:35.795805 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:35.797316 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.797465 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.797705 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.798436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:35.798800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:35.800443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:35.823280 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1367/2729] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:48:35.824351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:35.824475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62954067-fd77-4eda-b745-34186c764522 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:35.826264 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-62954067-fd77-4eda-b745-34186c764522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:35.838024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57fbf56-4b2b-47f6-9627-5836c9d3c48b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:35.848151 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:35.848847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.849097 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.849375 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.850167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:35.852483 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:35.863689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:35.895018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1368/2730] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:35 2022] POST /identity/v3/projects => generated 409 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:35.899167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:35.926262 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62954067-fd77-4eda-b745-34186c764522 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:35.988318 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518dbf5c-6531-4b43-8fc6-6985bd3fee32 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:35.995787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:35.996937 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.997277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:35.997620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.998081 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:35.998279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:35.998442 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:35.998789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.000132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.001203 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:36.002542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.015298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.017754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.030092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2731] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:36.035557 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1369/2732] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:35 2022] POST /identity/v3/users => generated 305 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:48:36.041225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.057686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9cf40e-54c1-4695-b699-cd0ff6e24f87 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.069254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.069949 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.070228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.070502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.071288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.073592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.089074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2733] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:36.093528 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.105809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3239a5e-6e4b-48d1-837b-d93043234890 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.111114 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.111836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.112165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.112443 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.113600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.114843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4dbf4-612c-4570-a933-b5de709a3826 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.129984 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1370/2734] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:36.131067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.143149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f935906-28fb-49e7-aa4c-210291c5a00c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.148639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.149860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.150331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.150819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] PATH_INFO: `/v3/projects/30b9cfb1f8334ca28c0fbfbb10169d44/users/c94fc7e9363643b8b8af377d8fdd5198/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.168092 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b9cfb1f8334ca28c0fbfbb10169d44, user_id=c94fc7e9363643b8b8af377d8fdd5198, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.171136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.198902 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2735] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:36 2022] PUT /identity/v3/projects/30b9cfb1f8334ca28c0fbfbb10169d44/users/c94fc7e9363643b8b8af377d8fdd5198/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:36.258642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.258907 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.259182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.291142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] MFA Rules not processed for user `c94fc7e9363643b8b8af377d8fdd5198`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:36.292496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:36.377973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1371/2736] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:36.382885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.451436 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c451dc20-2e3a-4acc-bda9-6b5ab50f6697 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.456618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.457287 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.457563 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.457831 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.458515 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.460467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.491116 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2737] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:36 2022] POST /identity/v3/projects => generated 409 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:36.492296 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.499436 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:36.504965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5a5ad-0aac-46ca-ad50-56d70ea6c122 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.567098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.567717 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.568000 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.568308 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.569030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.571087 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.604181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1372/2738] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:36 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:48:36.608077 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.620273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c65649-bb37-44bf-9b46-0cfcfed93173 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.628916 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.629603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.629884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.630160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.630957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.632716 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.645351 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2739] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:36.649086 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.659990 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386855e-9992-499b-9500-1f3082a9fdf4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.664965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.666143 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.666429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.666708 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] PATH_INFO: `/v3/projects/60463a48eec146e4bc18ec246830803c/users/c7db970d6a9f4f568ca340b17b90feec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.687288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=60463a48eec146e4bc18ec246830803c, user_id=c7db970d6a9f4f568ca340b17b90feec, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.687288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.724105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1373/2740] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:36 2022] PUT /identity/v3/projects/60463a48eec146e4bc18ec246830803c/users/c7db970d6a9f4f568ca340b17b90feec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:36.727334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.738857 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989155d3-aaba-4443-8241-2348adb2e54f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.753099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:36.816199 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.817393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.817393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.817618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.818459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.821443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdb96e-f1df-49d2-b82b-570e0eeba9f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.841345 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2741] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:36.846278 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:36.858068 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885267c-cb37-4702-94d3-3c6527758a8d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:36.863204 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:36.864142 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.864423 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.864698 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] PATH_INFO: `/v3/projects/60463a48eec146e4bc18ec246830803c/users/c7db970d6a9f4f568ca340b17b90feec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.866734 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] RBAC: Authorizing `identity:create_grant(project_id=60463a48eec146e4bc18ec246830803c, user_id=c7db970d6a9f4f568ca340b17b90feec, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:36.869479 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:36.882177 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1374/2742] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:36 2022] PUT /identity/v3/projects/60463a48eec146e4bc18ec246830803c/users/c7db970d6a9f4f568ca340b17b90feec/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:48:36.892891 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:36.893359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:36.893640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:36.920090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] MFA Rules not processed for user `c7db970d6a9f4f568ca340b17b90feec`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:36.921314 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:36.996705 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2743] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:37.008402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:37.010223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:37.073819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a766-d115-47ef-9fb4-81ab12c5467c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:37.078264 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:37.078921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:37.079200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:37.079472 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:37.089879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:37.091218 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:37.102856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1375/2744] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:37.306048 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:37.319785 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3dc08-38d8-42b2-acd8-cf3575405361 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:37.323867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:37.325254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:37.325651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:37.326012 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:37.411213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:37.412926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:37.423403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2745] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:37.795506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:37.796965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:37.842276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:37.844200 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:37.891405 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:37.898003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:37.909498 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ff864-a8b6-4c71-baa6-3979f413994f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:37.916814 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:37.917484 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:37.917769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:37.918064 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:37.919929 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:37.929025 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39484bff-d568-4f37-8e67-45508e451e5e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:37.933409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:37.934390 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:37.935002 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:37.935418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:37.935771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:37.936061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] PATH_INFO: `/v3/users/0a3a64d88e89402b907789c7f030e080` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:37.939083 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a3a64d88e89402b907789c7f030e080)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:37.939867 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:37.945945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1376/2746] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:37.997740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a3a64d88e89402b907789c7f030e080'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.007111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a3a64d88e89402b907789c7f030e080'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.007786 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a3a64d88e89402b907789c7f030e080'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.008453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a3a64d88e89402b907789c7f030e080'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.021556 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a3a64d88e89402b907789c7f030e080'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.036751 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2747] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:37 2022] DELETE /identity/v3/users/0a3a64d88e89402b907789c7f030e080 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:38.311727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:38.323729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206f2ac-1545-4c51-9c08-110600b7c87d service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:38.391123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:38.392016 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:38.392351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:38.392633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] PATH_INFO: `/v3/projects/dc2810000275432fb1337151b6883e90` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:38.393891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc2810000275432fb1337151b6883e90)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:38.395819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:38.455651 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc2810000275432fb1337151b6883e90', 'request_id': 'req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.463094 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc2810000275432fb1337151b6883e90', 'request_id': 'req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.470980 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc2810000275432fb1337151b6883e90', 'request_id': 'req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.487640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] The token cache is being invalidate because project dc2810000275432fb1337151b6883e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:38.487640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.488102 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.491141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1377/2748] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:38 2022] DELETE /identity/v3/projects/dc2810000275432fb1337151b6883e90 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:38.495184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:38.496905 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:38.512223 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1ab3d-4310-4e15-942c-36b28d8f0eb0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:38.582074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:38.582985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:38.583269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:38.583548 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] PATH_INFO: `/v3/users/09088c96d0b24f109c801a8e4fd1dbbf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:38.584860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09088c96d0b24f109c801a8e4fd1dbbf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:38.586810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:38.636757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09088c96d0b24f109c801a8e4fd1dbbf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.646066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09088c96d0b24f109c801a8e4fd1dbbf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.646774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09088c96d0b24f109c801a8e4fd1dbbf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.647398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09088c96d0b24f109c801a8e4fd1dbbf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.648709 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:38.664476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09088c96d0b24f109c801a8e4fd1dbbf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:38.679680 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2749] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:38 2022] DELETE /identity/v3/users/09088c96d0b24f109c801a8e4fd1dbbf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:38.862783 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:38.864778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:38.882065 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bfd4e-400a-413c-bd42-dfa660d9bee0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:38.955915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:38.957051 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:38.957285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:38.957683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] PATH_INFO: `/v3/projects/c5765604521448658675300905017afe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:38.958974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5765604521448658675300905017afe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:38.961213 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:38.990633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:39.025275 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5765604521448658675300905017afe', 'request_id': 'req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:39.033750 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5765604521448658675300905017afe', 'request_id': 'req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:39.046329 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5765604521448658675300905017afe', 'request_id': 'req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:39.068098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] The token cache is being invalidate because project c5765604521448658675300905017afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:39.068098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:39.068098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17a8bf39-586c-4f7a-9e1b-30cf723d0f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:39.080741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1378/2750] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:38 2022] DELETE /identity/v3/projects/c5765604521448658675300905017afe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:44.978755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:44.980636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:44.996713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1332a0e8-29da-440a-9d88-9cee7d2fdf1b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:45.063498 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:45.064418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:45.064776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:45.065049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] PATH_INFO: `/v3/users/c94fc7e9363643b8b8af377d8fdd5198` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:45.066298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94fc7e9363643b8b8af377d8fdd5198)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:45.068279 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:45.117079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94fc7e9363643b8b8af377d8fdd5198'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.126134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94fc7e9363643b8b8af377d8fdd5198'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.126830 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94fc7e9363643b8b8af377d8fdd5198'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.127655 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94fc7e9363643b8b8af377d8fdd5198'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.141249 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94fc7e9363643b8b8af377d8fdd5198'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.155372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2751] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:44 2022] DELETE /identity/v3/users/c94fc7e9363643b8b8af377d8fdd5198 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:45.160373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:45.160861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:45.160861 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:45.179538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:45.180772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:45.241833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1379/2752] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:48:45.251814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:45.253843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:45.318312 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ca0a2-ce34-488f-8a59-008cf0532f22 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:45.323828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:45.324532 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:45.324809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:45.325283 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:45.333872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:45.339260 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:45.342507 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:45.358973 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2753] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:45.630090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:45.631581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:45.644924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f771cec-262d-4d3f-b938-a1a1def50f01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:45.650570 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:45.651495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:45.651741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:45.652021 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] PATH_INFO: `/v3/projects/30b9cfb1f8334ca28c0fbfbb10169d44` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:45.653398 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b9cfb1f8334ca28c0fbfbb10169d44)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:45.655360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:45.706985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b9cfb1f8334ca28c0fbfbb10169d44', 'request_id': 'req-0b8dc538-d351-4bc6-be23-b370e5137e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.712552 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b9cfb1f8334ca28c0fbfbb10169d44', 'request_id': 'req-0b8dc538-d351-4bc6-be23-b370e5137e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.721826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b9cfb1f8334ca28c0fbfbb10169d44', 'request_id': 'req-0b8dc538-d351-4bc6-be23-b370e5137e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.739653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] The token cache is being invalidate because project 30b9cfb1f8334ca28c0fbfbb10169d44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:45.739920 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.740874 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.743174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1380/2754] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:45 2022] DELETE /identity/v3/projects/30b9cfb1f8334ca28c0fbfbb10169d44 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:45.747332 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:45.759153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30ba47-8da1-4a95-979c-d275e2cbc9b8 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:45.818838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:45.819703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:45.819976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:45.820269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] PATH_INFO: `/v3/users/c7db970d6a9f4f568ca340b17b90feec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:45.821488 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7db970d6a9f4f568ca340b17b90feec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:45.823416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:45.875829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7db970d6a9f4f568ca340b17b90feec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.884376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7db970d6a9f4f568ca340b17b90feec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.884995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7db970d6a9f4f568ca340b17b90feec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.885615 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7db970d6a9f4f568ca340b17b90feec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.897800 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f1df15c-b140-4d89-8fef-6fba39aa0c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7db970d6a9f4f568ca340b17b90feec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:45.909732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2755] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:45 2022] DELETE /identity/v3/users/c7db970d6a9f4f568ca340b17b90feec => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:46.190550 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.192122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:46.205947 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dc538-d351-4bc6-be23-b370e5137e03 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.236791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:46.263197 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.263988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.264299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.264565 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] PATH_INFO: `/v3/projects/60463a48eec146e4bc18ec246830803c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.265723 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60463a48eec146e4bc18ec246830803c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.267685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.313856 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60463a48eec146e4bc18ec246830803c', 'request_id': 'req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:46.319772 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60463a48eec146e4bc18ec246830803c', 'request_id': 'req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:46.328670 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60463a48eec146e4bc18ec246830803c', 'request_id': 'req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:46.340022 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] The token cache is being invalidate because project 60463a48eec146e4bc18ec246830803c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:46.340348 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:46.340999 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1e60b76a-7f20-42bb-89c8-754a3d73a5c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:46.342573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1381/2756] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:46 2022] DELETE /identity/v3/projects/60463a48eec146e4bc18ec246830803c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:46.348635 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.349160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.349160 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.367910 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:46.369360 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:46.433993 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2757] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:46.442653 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.443006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.443380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.471559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:46.472825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:46.482210 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1382/2758] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:46.486639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.496676 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780303d-b75d-4d5c-a958-8069e59465fe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.502592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.503280 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.503554 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.503806 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.504568 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:48:46.504872 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.506538 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.515704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:46.519441 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2759] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:48:46.523727 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.533082 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce952c-e19c-4b3c-a595-b4177dde9ee0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.538007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.538645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.538919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.539185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.539865 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.541798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.570932 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1383/2760] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:48:46 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:46.575151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.586180 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66af93-2c46-4918-9a2a-e07b30d3f671 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.645385 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.646111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.646393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.646668 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.647347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.649317 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.679918 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2761] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:48:46 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:48:46.684537 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.696373 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c634fe-b666-4c30-ab55-60613072d137 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.701428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.702098 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.702378 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.702648 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.703401 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.705050 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.718224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1384/2762] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:46.722489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.732957 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fbc55-a9fc-4130-b16e-063d7a8c38b4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.738567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.739273 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.739550 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.739820 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.740647 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.742224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf14e80-e9eb-4ef8-959b-acd274e52af9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.755309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2763] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:48:46.759915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.771223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc079201-f790-4416-bc24-cbe40144884c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:46.777665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:46.778559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.778832 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.779106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] PATH_INFO: `/v3/projects/bfb8c0a82ecf491484bcb896915af454/users/daf2b5175f194089a6cdbd1ce7a4b822/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.795904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb8c0a82ecf491484bcb896915af454, user_id=daf2b5175f194089a6cdbd1ce7a4b822, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:46.798665 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:46.815392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1385/2764] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:48:46 2022] PUT /identity/v3/projects/bfb8c0a82ecf491484bcb896915af454/users/daf2b5175f194089a6cdbd1ce7a4b822/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:46.826790 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:46.827067 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:46.827395 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:46.856148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] MFA Rules not processed for user `daf2b5175f194089a6cdbd1ce7a4b822`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:46.857355 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:46.938277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2765] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:46.950419 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:46.952497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:47.017234 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb67e58-ea92-4f32-ac98-5ec540317fb2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:47.022046 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:47.022633 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:47.022912 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:47.023186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:47.033195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:47.034647 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add8711-8d18-4566-85c0-2839bb358387 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:47.046848 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1386/2766] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:47.074879 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:47.149512 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a90874-67e1-46f6-99ba-68ed9d005e18 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:47.157963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:47.158956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:47.159230 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:47.159511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] PATH_INFO: `/v3/users/daf2b5175f194089a6cdbd1ce7a4b822` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:47.160856 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf2b5175f194089a6cdbd1ce7a4b822)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:47.162860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:47.174359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:47.210975 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf2b5175f194089a6cdbd1ce7a4b822'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.219013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf2b5175f194089a6cdbd1ce7a4b822'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.219666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf2b5175f194089a6cdbd1ce7a4b822'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.220326 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf2b5175f194089a6cdbd1ce7a4b822'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.233703 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf2b5175f194089a6cdbd1ce7a4b822'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.246358 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2767] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:48:47 2022] DELETE /identity/v3/users/daf2b5175f194089a6cdbd1ce7a4b822 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:47.251380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:47.251871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:47.252073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:47.258434 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:47.273305 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:47.274452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:47.337497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1387/2768] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:47.350149 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:47.351816 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:47.413627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011bd0bb-15cb-4bfa-8203-d1245424c318 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:47.418109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:47.418758 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:47.419039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:47.419307 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:47.429376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:47.430829 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:47.440346 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2769] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:47.718589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:47.720071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:47.733007 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ccd5b-1076-437f-9aae-4e7996434d9b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:47.739131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:47.740013 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:47.740319 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:47.740592 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] PATH_INFO: `/v3/projects/bfb8c0a82ecf491484bcb896915af454` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:47.741904 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb8c0a82ecf491484bcb896915af454)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:47.743808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:47.793546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb8c0a82ecf491484bcb896915af454', 'request_id': 'req-a720f4f3-a4cc-4d67-b3de-0846174a1b86'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.798813 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb8c0a82ecf491484bcb896915af454', 'request_id': 'req-a720f4f3-a4cc-4d67-b3de-0846174a1b86'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.807574 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb8c0a82ecf491484bcb896915af454', 'request_id': 'req-a720f4f3-a4cc-4d67-b3de-0846174a1b86'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.819389 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] The token cache is being invalidate because project bfb8c0a82ecf491484bcb896915af454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:48:47.819674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.820395 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:48:47.822133 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1388/2770] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:48:47 2022] DELETE /identity/v3/projects/bfb8c0a82ecf491484bcb896915af454 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:48:57.436483 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=10,keystone:SELECT=48,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:57.816350 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:21.924656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:21.926233 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:22.014573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f8d8-3919-4423-a240-bf770e877f96 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:22.095443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:22.096416 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:22.096834 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:22.097229 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:22.099334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:22.114701 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:22.116646 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:22.132755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2771] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:30.592824 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:30.594593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:30.658862 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720f4f3-a4cc-4d67-b3de-0846174a1b86 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:30.663905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:30.665674 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:30.665955 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:30.666227 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] PATH_INFO: `/v3/users/0f8105d1de0d45218576d74a2ba4abe4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:30.667546 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8105d1de0d45218576d74a2ba4abe4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:30.669825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:30.736600 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8105d1de0d45218576d74a2ba4abe4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:30.747507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8105d1de0d45218576d74a2ba4abe4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:30.748345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8105d1de0d45218576d74a2ba4abe4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:30.755884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8105d1de0d45218576d74a2ba4abe4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:30.756500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b9bb3fd9-192c-4457-9b28-99e6c9ce4cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8105d1de0d45218576d74a2ba4abe4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:30.773825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1389/2772] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:30 2022] DELETE /identity/v3/users/0f8105d1de0d45218576d74a2ba4abe4 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:30.954266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:30.969152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272cb62-e005-448d-afa7-ef2668e2021e service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.030884 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.031880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.032185 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.032459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] PATH_INFO: `/v3/projects/a30edd891b96456fbba6409a815f5fd8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.033693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a30edd891b96456fbba6409a815f5fd8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.035625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.073907 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:31.092313 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a30edd891b96456fbba6409a815f5fd8', 'request_id': 'req-35098b6e-1784-448f-810b-4847886db33a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:31.097750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a30edd891b96456fbba6409a815f5fd8', 'request_id': 'req-35098b6e-1784-448f-810b-4847886db33a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:31.106254 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a30edd891b96456fbba6409a815f5fd8', 'request_id': 'req-35098b6e-1784-448f-810b-4847886db33a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:31.117456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] The token cache is being invalidate because project a30edd891b96456fbba6409a815f5fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:31.117743 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:31.118455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-35098b6e-1784-448f-810b-4847886db33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:31.120151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2773] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:30 2022] DELETE /identity/v3/projects/a30edd891b96456fbba6409a815f5fd8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:31.125460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.125807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.126125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.144608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:31.145811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:31.215200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1390/2774] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:31.223782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.224094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.224376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.243111 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:31.244339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:31.257150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2775] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:31.261520 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.271888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3b865-0e9c-4469-b12c-af322d8b5693 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.277775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.278430 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.278712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.278985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.279714 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:49:31.280019 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.281789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.293656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1391/2776] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:49:31.298614 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.307741 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3606f860-8b4a-4b94-963c-6eceb07e300d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.312592 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.313241 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.313517 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.313784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.314472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.316459 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.345094 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2777] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:31.349680 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.360317 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f4ea2-a6ca-4ccb-a43c-fdb06bbc5643 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.364849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:31.420879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.421548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.421867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.422121 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.422798 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.424822 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.444807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1392/2778] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:31.449002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.459876 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f8389-fe35-465a-a23f-b23739d65498 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.465658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.466319 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.466607 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.466863 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.467585 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.469282 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.481611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2779] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:31.485644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.496452 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21c7dd-cab9-4f47-93c1-d6d3d97042a4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.503004 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.503663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.503941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.504244 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.504966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.506567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cab6bc-b11b-4455-bee3-65a6fcfbd938 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.517685 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1393/2780] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:31.521925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.532909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25f993-7dce-4a69-8a5f-8d0070f4fbd2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.537611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.538782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.539059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.539327 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] PATH_INFO: `/v3/projects/a827835d40404f4991bf181fdef634e1/users/90fcdf191d204253bb0d5f779ad52a51/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.553928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a827835d40404f4991bf181fdef634e1, user_id=90fcdf191d204253bb0d5f779ad52a51, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.556736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.570849 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2781] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:31 2022] PUT /identity/v3/projects/a827835d40404f4991bf181fdef634e1/users/90fcdf191d204253bb0d5f779ad52a51/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:31.583690 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.584101 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.584541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.615291 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] MFA Rules not processed for user `90fcdf191d204253bb0d5f779ad52a51`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:31.616562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:31.684212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1394/2782] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:31.688575 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.754164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243d7e0-dd74-4220-a312-9c914aae84a9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.759135 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.759792 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.760119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.760384 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.761064 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.763289 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.790300 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2783] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:31.794294 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.805890 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c98e-97a5-4942-b0c5-220f6db02f24 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.869243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.869918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.870193 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.870462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.871161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.873126 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.895444 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1395/2784] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:31 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:49:31.899588 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.911181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74301c85-7847-41bf-9194-728bee660ee7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.916744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.917431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.917710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.918000 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.919004 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.921494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.933088 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2785] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:31.937208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.943880 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:31.948185 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71b49d7-3606-4163-906f-8c093383626f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.954237 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.954898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.955180 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.955459 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:31.956221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:31.957825 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0b0fd-7f9c-4b19-94f9-dd41423ade27 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:31.968076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1396/2786] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:31.972382 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:31.982531 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ded3a-815d-47cc-b1c7-9c043262f53d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:31.987787 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:31.988708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:31.988985 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:31.989285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] PATH_INFO: `/v3/projects/e5ca02c0fc494ad69bea85d6d051b8f5/users/4f5e0dffb8a14486a1b97f7e3b2bbd19/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.005713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ca02c0fc494ad69bea85d6d051b8f5, user_id=4f5e0dffb8a14486a1b97f7e3b2bbd19, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.008418 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.020153 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:32.026125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2787] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:31 2022] PUT /identity/v3/projects/e5ca02c0fc494ad69bea85d6d051b8f5/users/4f5e0dffb8a14486a1b97f7e3b2bbd19/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:32.036742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.037034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.037311 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.065199 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] MFA Rules not processed for user `4f5e0dffb8a14486a1b97f7e3b2bbd19`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:32.066326 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:32.133821 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1397/2788] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:32.138776 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.205060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fef51-dad6-4d57-af4a-1f5d1ddfc799 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.211035 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.211692 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.211956 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.212277 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.213010 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.214935 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.242352 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2789] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:32 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:32.246392 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.257194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f391be-3301-4c28-89dd-47db7eae5227 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.316891 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.317501 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.317794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.318071 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.318801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.320870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.344715 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1398/2790] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:32 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:32.348939 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.359728 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c67c3-1f10-4a70-b6db-ae0e17a50478 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.364541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.365219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.365535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.365812 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.366577 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.368250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.381754 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2791] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:32.386796 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.402287 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160b913-945a-4efb-97cd-edb7b2c17023 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.409299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.410226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.410482 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.410755 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] PATH_INFO: `/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6/users/62f83b529fad4cb99e03ca42060005fb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.427590 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c611d5fa2b14b22ad1e689d7526e5d6, user_id=62f83b529fad4cb99e03ca42060005fb, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.430285 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.445938 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1399/2792] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:32 2022] PUT /identity/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6/users/62f83b529fad4cb99e03ca42060005fb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:32.450148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.461561 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d0b14-2644-4108-b861-563d61c697f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.521678 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.522376 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.522704 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.522996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.523710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.525431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb088c-81ae-4f7c-a6e9-00c695b5b6c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.536386 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2793] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:32.540924 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.551950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80cbbdf-924e-4e04-92f6-1de87353784c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.556864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.557752 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.558024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.558292 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] PATH_INFO: `/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6/users/62f83b529fad4cb99e03ca42060005fb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.560331 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c611d5fa2b14b22ad1e689d7526e5d6, user_id=62f83b529fad4cb99e03ca42060005fb, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.562981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.574909 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1400/2794] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:32 2022] PUT /identity/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6/users/62f83b529fad4cb99e03ca42060005fb/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:32.585391 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.585684 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.585961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.616225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] MFA Rules not processed for user `62f83b529fad4cb99e03ca42060005fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:32.617446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:32.677576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:32.687855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2795] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:32.698438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:32.700460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:32.742414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:32.761554 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ca898-60ca-4440-9cfa-207d89e8a56f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:32.765867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:32.766497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:32.766776 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:32.767034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:32.776298 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:32.777746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:32.787232 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1401/2796] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:33.318430 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:33.320063 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:33.333349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36ddf0-2262-474c-8ceb-31826542b164 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:33.337778 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:33.338431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:33.338712 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:33.338986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:33.412267 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:33.413971 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:33.427367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2797] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:37.514368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:37.523615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d2a84-20e1-4e18-ada1-2c8467fa0fbc service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:37.527538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:37.528226 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:37.528507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:37.528777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:37.587474 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:37.588981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb4df31-cb0a-4680-8186-42ad0b0bc465 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:37.598365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1402/2798] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:39.728873 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:39.796854 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139535c0-24a0-4cbb-83f5-cd6c9c31bac7 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:39.801611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:39.802511 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:39.802793 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:39.803065 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] PATH_INFO: `/v3/users/90fcdf191d204253bb0d5f779ad52a51` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:39.804306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90fcdf191d204253bb0d5f779ad52a51)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:39.806268 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:39.870049 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90fcdf191d204253bb0d5f779ad52a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:39.879148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90fcdf191d204253bb0d5f779ad52a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:39.879799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90fcdf191d204253bb0d5f779ad52a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:39.880597 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90fcdf191d204253bb0d5f779ad52a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:39.893651 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90fcdf191d204253bb0d5f779ad52a51'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:39.906997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2799] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:39 2022] DELETE /identity/v3/users/90fcdf191d204253bb0d5f779ad52a51 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:39.912659 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:39.913149 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:39.913267 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:39.932694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:39.934018 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:39.996981 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1403/2800] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:40.007296 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:40.009838 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:40.027219 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:40.071074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab7d14-5bcc-4f02-9587-1a8ffcfc5476 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:40.077669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:40.078685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:40.079059 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:40.079448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:40.091780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:40.093325 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:40.103349 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2801] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:40.402312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:40.403762 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:40.416088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc212cff-aa9d-46b1-91b8-4d262f958802 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:40.421742 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:40.422644 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:40.422925 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:40.423202 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] PATH_INFO: `/v3/projects/a827835d40404f4991bf181fdef634e1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:40.424542 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a827835d40404f4991bf181fdef634e1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:40.426519 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:40.483368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a827835d40404f4991bf181fdef634e1', 'request_id': 'req-8cc3567a-1256-4b93-8286-2999c0e5c3fe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.488569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a827835d40404f4991bf181fdef634e1', 'request_id': 'req-8cc3567a-1256-4b93-8286-2999c0e5c3fe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.497854 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a827835d40404f4991bf181fdef634e1', 'request_id': 'req-8cc3567a-1256-4b93-8286-2999c0e5c3fe'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.506100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:40.510889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] The token cache is being invalidate because project a827835d40404f4991bf181fdef634e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:40.511182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.512054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.514611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1404/2802] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:40 2022] DELETE /identity/v3/projects/a827835d40404f4991bf181fdef634e1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:40.518997 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:40.529310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e07ddf-c52c-44d5-b42f-ac39e5674e0f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:40.588075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:40.589186 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:40.589466 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:40.589736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] PATH_INFO: `/v3/users/4f5e0dffb8a14486a1b97f7e3b2bbd19` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:40.590933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5e0dffb8a14486a1b97f7e3b2bbd19)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:40.592920 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:40.639842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5e0dffb8a14486a1b97f7e3b2bbd19'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.647848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5e0dffb8a14486a1b97f7e3b2bbd19'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.648497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5e0dffb8a14486a1b97f7e3b2bbd19'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.649074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5e0dffb8a14486a1b97f7e3b2bbd19'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.662639 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5e0dffb8a14486a1b97f7e3b2bbd19'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:40.677489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2803] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:40 2022] DELETE /identity/v3/users/4f5e0dffb8a14486a1b97f7e3b2bbd19 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:40.967506 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:40.969672 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:40.983428 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3567a-1256-4b93-8286-2999c0e5c3fe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:41.043696 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:41.044513 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.044789 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.045042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] PATH_INFO: `/v3/projects/e5ca02c0fc494ad69bea85d6d051b8f5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.046212 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ca02c0fc494ad69bea85d6d051b8f5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:41.048181 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:41.096293 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ca02c0fc494ad69bea85d6d051b8f5', 'request_id': 'req-7ff55649-a50d-461d-8388-5697dd444339'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.102186 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ca02c0fc494ad69bea85d6d051b8f5', 'request_id': 'req-7ff55649-a50d-461d-8388-5697dd444339'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.111988 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ca02c0fc494ad69bea85d6d051b8f5', 'request_id': 'req-7ff55649-a50d-461d-8388-5697dd444339'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.123778 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] The token cache is being invalidate because project e5ca02c0fc494ad69bea85d6d051b8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:41.124090 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.124726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.126263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1405/2804] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:40 2022] DELETE /identity/v3/projects/e5ca02c0fc494ad69bea85d6d051b8f5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:41.130527 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:41.131969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:41.149693 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef613b-1bae-4053-b037-693991d742b0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:41.183986 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:41.217673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:41.218573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.218846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.219136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] PATH_INFO: `/v3/users/62f83b529fad4cb99e03ca42060005fb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.220367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f83b529fad4cb99e03ca42060005fb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:41.222329 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:41.270072 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f83b529fad4cb99e03ca42060005fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.278030 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f83b529fad4cb99e03ca42060005fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.278650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f83b529fad4cb99e03ca42060005fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.279321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f83b529fad4cb99e03ca42060005fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.290984 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-03db4980-434e-420f-b39c-2f8465629bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f83b529fad4cb99e03ca42060005fb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.303710 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2805] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:41 2022] DELETE /identity/v3/users/62f83b529fad4cb99e03ca42060005fb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:41.594787 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:41.596677 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:41.611991 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55649-a50d-461d-8388-5697dd444339 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:41.671403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:41.672320 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.672602 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.672878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] PATH_INFO: `/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.674060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c611d5fa2b14b22ad1e689d7526e5d6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:41.676057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:41.715223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:41.728871 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c611d5fa2b14b22ad1e689d7526e5d6', 'request_id': 'req-9eeab3be-6593-4437-83ce-f18850d90212'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.741333 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c611d5fa2b14b22ad1e689d7526e5d6', 'request_id': 'req-9eeab3be-6593-4437-83ce-f18850d90212'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.750112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c611d5fa2b14b22ad1e689d7526e5d6', 'request_id': 'req-9eeab3be-6593-4437-83ce-f18850d90212'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.761729 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] The token cache is being invalidate because project 3c611d5fa2b14b22ad1e689d7526e5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:41.762014 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.762966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9eeab3be-6593-4437-83ce-f18850d90212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:41.764579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1406/2806] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:41 2022] DELETE /identity/v3/projects/3c611d5fa2b14b22ad1e689d7526e5d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:41.770765 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.771138 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.771417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.790582 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:41.791805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:41.856540 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2807] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:41.865793 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.866085 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.866365 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.885906 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:41.887125 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:41.896368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1407/2808] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:49:41.901001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:41.911091 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5167db-a218-468b-ae24-d529378d0219 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:41.916266 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:41.917246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.917534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.917813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.918544 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:49:41.918851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:41.920502 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:41.933129 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2809] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:49:41.937962 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:41.948580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2873d085-14af-48c2-b9c1-9c773b1f2f64 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:41.957252 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:41.958216 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:41.958619 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:41.958996 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:41.960084 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:41.963077 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:41.991357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1408/2810] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:41 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:41.996054 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.006611 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba798dab-a669-4568-a582-a7b12e0aaa60 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.049736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:42.068610 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.069298 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.069579 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.069848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.070547 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.072509 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.096074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2811] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:41 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:42.101117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.112643 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff8c8e-19fa-4c9f-9f94-9ce223ec9432 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.117615 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.118277 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.118562 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.118844 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.119563 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.121358 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.132100 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1409/2812] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:42.135813 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.146998 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83861517-1bb9-483f-a305-578c45d757d4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.152477 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.153141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.153419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.153696 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.154423 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.156047 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3491d2-17af-41c8-9644-da2524778174 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.165954 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2813] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:42.170969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.186414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ffed2-7162-4519-aa28-1d4580ae7254 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.195036 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.196379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.196662 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.197038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] PATH_INFO: `/v3/projects/5f82a1a651bf4d4abd60981c7ee361e4/users/627f1be4875347fcbe8775a8ade30c22/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.213974 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f82a1a651bf4d4abd60981c7ee361e4, user_id=627f1be4875347fcbe8775a8ade30c22, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.216744 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.232967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1410/2814] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:42 2022] PUT /identity/v3/projects/5f82a1a651bf4d4abd60981c7ee361e4/users/627f1be4875347fcbe8775a8ade30c22/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:49:42.244685 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.244979 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.245294 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.273166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] MFA Rules not processed for user `627f1be4875347fcbe8775a8ade30c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:42.274392 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:42.343821 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2815] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:49:42.356569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.358572 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:42.421993 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526742a-5ac9-48b4-a1f1-831927504bd1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.426809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.427507 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.428023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.428367 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.439617 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.441306 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3fac00-7bf5-488a-a9ab-69554f4ea499 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.451496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1411/2816] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:42.472528 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.544146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b6e7d-4615-498d-b49a-44a74b85c297 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.549367 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.550250 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.550529 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.550799 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] PATH_INFO: `/v3/users/627f1be4875347fcbe8775a8ade30c22` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.552337 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=627f1be4875347fcbe8775a8ade30c22)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.554285 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.617140 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627f1be4875347fcbe8775a8ade30c22'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:42.621134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:42.627535 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627f1be4875347fcbe8775a8ade30c22'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:42.628494 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627f1be4875347fcbe8775a8ade30c22'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:42.629462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627f1be4875347fcbe8775a8ade30c22'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:42.642356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627f1be4875347fcbe8775a8ade30c22'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:42.654675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2817] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:42 2022] DELETE /identity/v3/users/627f1be4875347fcbe8775a8ade30c22 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:42.659543 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.659829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.660132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.678297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:42.679564 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:42.733788 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:42.744620 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1412/2818] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:42.756897 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:42.758814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:42.821727 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff93262-acdb-48a3-a282-c0cc0b8c5e1a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:42.826090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:42.826740 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:42.827028 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:42.827309 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:42.837139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:42.838599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9a369-d277-409c-9651-39883c2b93d1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:42.848851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2819] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:43.166849 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.168630 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:43.180823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b71f2-2162-4dae-abc0-fd17cd95e752 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.185870 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.186749 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.187032 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.187343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] PATH_INFO: `/v3/projects/5f82a1a651bf4d4abd60981c7ee361e4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.188580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f82a1a651bf4d4abd60981c7ee361e4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.190495 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.247790 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f82a1a651bf4d4abd60981c7ee361e4', 'request_id': 'req-f6d4f417-b959-4de9-a98b-414f2cbaed4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:43.253578 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f82a1a651bf4d4abd60981c7ee361e4', 'request_id': 'req-f6d4f417-b959-4de9-a98b-414f2cbaed4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:43.263368 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f82a1a651bf4d4abd60981c7ee361e4', 'request_id': 'req-f6d4f417-b959-4de9-a98b-414f2cbaed4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:43.274889 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] The token cache is being invalidate because project 5f82a1a651bf4d4abd60981c7ee361e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:43.275168 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:43.275815 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f6d4f417-b959-4de9-a98b-414f2cbaed4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:43.277584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1413/2820] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:43 2022] DELETE /identity/v3/projects/5f82a1a651bf4d4abd60981c7ee361e4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:43.282715 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.283306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.283306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.301644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:43.302839 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:43.364152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2821] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:43.374477 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.374823 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.375163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.399746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:43.401011 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:43.410361 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1414/2822] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:43.416731 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.432479 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d497f-53c8-4133-bdcb-5849779ba4ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.438288 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.438940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.439214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.439468 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.440228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:49:43.440524 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.442146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.454320 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2823] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:49:43.458757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.468826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa908ec-039b-4439-9d00-f6e2f76f0408 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.474150 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.474800 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.475076 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.475344 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.476024 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.477954 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.506910 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1415/2824] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:43.512885 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.523318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af05ad-774b-4da4-8bb8-88982c693a15 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.583453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.584189 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.584472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.584745 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.585431 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.587370 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.594120 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:43.609700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2825] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:43.614261 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.625131 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6a9dd-de94-456f-b5d5-521e7a907bde admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.630807 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.631492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.631766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.632049 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.632771 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.634371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.645853 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1416/2826] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:43.649900 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.660076 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398a9c5-a202-450a-b1b5-3c07e4ae63cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.664974 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.665679 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.665964 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.666239 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.667099 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.668914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7f01d-e6ed-4f32-8173-f26ce51e0c4d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.679109 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2827] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:43.683581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.693720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e158de4-c170-4be4-b2fc-ab00ca91bd1f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.698741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.699569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.699847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.700141 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] PATH_INFO: `/v3/projects/510fcfd4b27a415d8dc791ad8d79e938/users/5628317a2b6845b19bd7d7213fb55eec/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.714791 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510fcfd4b27a415d8dc791ad8d79e938, user_id=5628317a2b6845b19bd7d7213fb55eec, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.717523 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.734836 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1417/2828] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:43 2022] PUT /identity/v3/projects/510fcfd4b27a415d8dc791ad8d79e938/users/5628317a2b6845b19bd7d7213fb55eec/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:49:43.750523 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.750908 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.751334 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.782990 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] MFA Rules not processed for user `5628317a2b6845b19bd7d7213fb55eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:43.784141 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:43.851146 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2829] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:43.856211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.927167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f33d91-b5da-4790-b9e4-844ea384fec1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:43.934885 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:43.935536 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:43.935816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:43.936109 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:43.936794 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:43.938701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:43.966939 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1418/2830] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:43.971082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:43.982620 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25de82d-33f3-4535-8c88-6aa1400c0ea9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.041096 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.041660 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.042130 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.042387 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.043095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.045060 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.065904 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2831] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:43 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:44.069930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:44.076908 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:44.080775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f46e1-78bf-4655-88b7-de2d9da4569f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.086809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.087840 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.088156 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.088486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.089616 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.091535 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.102223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1419/2832] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:44.106625 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:44.117115 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f0367-2aa5-4da8-9984-7378c19b7eba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.123310 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.124244 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.124501 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.124772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] PATH_INFO: `/v3/projects/d2360a75ec344d6cbbca6fe1fee74759/users/d2231396850f46adbd8686b6a6bc51a8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.140455 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2360a75ec344d6cbbca6fe1fee74759, user_id=d2231396850f46adbd8686b6a6bc51a8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.143131 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.159042 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2833] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:44 2022] PUT /identity/v3/projects/d2360a75ec344d6cbbca6fe1fee74759/users/d2231396850f46adbd8686b6a6bc51a8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:44.163642 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:44.179608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b19d38-1470-4da9-8c69-6f93a2f05aa8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.242613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.243281 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.243567 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.243843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.244596 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.246460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838714d-c235-4ca8-aa44-ceac210b656f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.256349 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1420/2834] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:49:44.260753 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:44.271448 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc195a73-805b-4a92-8a64-3cdb5df535c4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.276654 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.277818 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.278098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.278372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] PATH_INFO: `/v3/projects/d2360a75ec344d6cbbca6fe1fee74759/users/d2231396850f46adbd8686b6a6bc51a8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.280454 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2360a75ec344d6cbbca6fe1fee74759, user_id=d2231396850f46adbd8686b6a6bc51a8, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.283121 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.294026 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2835] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:44 2022] PUT /identity/v3/projects/d2360a75ec344d6cbbca6fe1fee74759/users/d2231396850f46adbd8686b6a6bc51a8/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:49:44.304208 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.304496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.304769 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.332741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] MFA Rules not processed for user `d2231396850f46adbd8686b6a6bc51a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:44.333945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:44.405921 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1421/2836] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:44.417356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:44.419842 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:44.445870 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:44.484791 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec3068-eab5-4063-a1e5-23d42ea06748 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:44.489443 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:44.490098 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:44.490375 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:44.490645 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:44.548261 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:44.549636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16da6b-88ea-4807-b8e3-f083c67a856f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:44.558427 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2837] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:47.131898 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:47.194687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809a663-365d-49a4-a86b-5f175a6c02dd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:47.199766 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:47.200663 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:47.200943 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:47.201219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] PATH_INFO: `/v3/users/5628317a2b6845b19bd7d7213fb55eec` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:47.202573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5628317a2b6845b19bd7d7213fb55eec)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:47.204608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:47.220737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:47.253720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5628317a2b6845b19bd7d7213fb55eec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.262038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5628317a2b6845b19bd7d7213fb55eec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.262968 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5628317a2b6845b19bd7d7213fb55eec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.269826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5628317a2b6845b19bd7d7213fb55eec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.270469 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5628317a2b6845b19bd7d7213fb55eec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.288728 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1422/2838] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:47 2022] DELETE /identity/v3/users/5628317a2b6845b19bd7d7213fb55eec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:47.293656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:47.294201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:47.294201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:47.314117 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:47.315318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:47.376398 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2839] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:47.386089 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:47.388127 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:47.448965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ea747-bdee-4647-a32e-234b37cc67d3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:47.453225 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:47.453858 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:47.454152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:47.454409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:47.463816 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:47.465268 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:47.474641 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1423/2840] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:47.754328 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:47.756228 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:47.774866 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea95a-aa5e-4ac6-872c-c0c56e3b5aaa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:47.782178 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:47.783340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:47.783652 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:47.784079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] PATH_INFO: `/v3/projects/510fcfd4b27a415d8dc791ad8d79e938` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:47.785598 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510fcfd4b27a415d8dc791ad8d79e938)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:47.788061 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:47.843125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510fcfd4b27a415d8dc791ad8d79e938', 'request_id': 'req-d5df167c-22f9-4422-9300-cd007d81bcc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.849472 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510fcfd4b27a415d8dc791ad8d79e938', 'request_id': 'req-d5df167c-22f9-4422-9300-cd007d81bcc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.857195 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510fcfd4b27a415d8dc791ad8d79e938', 'request_id': 'req-d5df167c-22f9-4422-9300-cd007d81bcc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.864356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:47.869874 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] The token cache is being invalidate because project 510fcfd4b27a415d8dc791ad8d79e938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:47.870497 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.871216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:47.873636 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2841] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:47 2022] DELETE /identity/v3/projects/510fcfd4b27a415d8dc791ad8d79e938 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:47.877809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:47.887905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6514d-1429-4f83-ad6b-6e6d24507787 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:47.946045 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:47.946936 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:47.947342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:47.947622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] PATH_INFO: `/v3/users/d2231396850f46adbd8686b6a6bc51a8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:47.949119 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2231396850f46adbd8686b6a6bc51a8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:47.952219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.004200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:48.011028 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2231396850f46adbd8686b6a6bc51a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.019830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2231396850f46adbd8686b6a6bc51a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.020508 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2231396850f46adbd8686b6a6bc51a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.027667 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2231396850f46adbd8686b6a6bc51a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.028343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-798a5daf-9e6b-4044-a342-1fd948598e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2231396850f46adbd8686b6a6bc51a8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.045514 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1424/2842] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:49:47 2022] DELETE /identity/v3/users/d2231396850f46adbd8686b6a6bc51a8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:48.341090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.342623 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:48.358700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df167c-22f9-4422-9300-cd007d81bcc8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.417256 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.418125 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.418404 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.418673 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] PATH_INFO: `/v3/projects/d2360a75ec344d6cbbca6fe1fee74759` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.419917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2360a75ec344d6cbbca6fe1fee74759)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.421869 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.468782 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2360a75ec344d6cbbca6fe1fee74759', 'request_id': 'req-88403ab7-8872-4415-b1bf-ea21e52e8130'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.474246 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2360a75ec344d6cbbca6fe1fee74759', 'request_id': 'req-88403ab7-8872-4415-b1bf-ea21e52e8130'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.482164 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2360a75ec344d6cbbca6fe1fee74759', 'request_id': 'req-88403ab7-8872-4415-b1bf-ea21e52e8130'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.493234 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] The token cache is being invalidate because project d2360a75ec344d6cbbca6fe1fee74759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:49:48.493508 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.494139 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88403ab7-8872-4415-b1bf-ea21e52e8130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:49:48.495843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2843] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:49:48 2022] DELETE /identity/v3/projects/d2360a75ec344d6cbbca6fe1fee74759 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:48.501888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.502402 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.502500 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.520986 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:48.522192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:48.587573 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1425/2844] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:48.597321 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.597730 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.598003 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.617848 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:48.619086 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:48.628762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2845] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:48.633691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.647902 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f4a10-14e8-45dd-aeb9-a590a7ec2e5c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.654694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.655325 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.655601 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.655867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.656640 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:49:48.656935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.658579 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.671242 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1426/2846] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:49:48.675732 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.685541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56cfc4-7f33-41bc-b8a9-92616d8799c5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.691674 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.692394 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.692672 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.692945 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.693691 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.695675 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.724744 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2847] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:49:48 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:48.729060 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.739545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb528e-5d1b-4369-b25d-944c5e042aea admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.797691 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.798354 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.798628 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.798901 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.799559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.801538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.823155 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1427/2848] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:49:48 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:49:48.831005 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.848627 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7cc2df-3bb3-44b0-9a51-889c497b3c18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.856490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.857301 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.857576 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.857850 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.858583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.860284 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.871996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2849] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:48.876560 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.899403 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc07d2-6b58-4aa2-a027-f7d523c276b4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.905967 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.906882 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.907384 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.907799 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.909017 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.911645 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ab5d3-584a-4c03-b8d5-de85c760f378 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.923351 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1428/2850] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:49:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:49:48.928066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:48.939825 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b382f6-b39f-4fb4-b806-44a7f2030d9c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:48.947940 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:48.949383 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.949666 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.949946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] PATH_INFO: `/v3/projects/f4c0c46c0d984a7fb3e10b0c1d8f2832/users/f7da249ee0ea4fe38db3f8ca11191225/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:48.968402 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c0c46c0d984a7fb3e10b0c1d8f2832, user_id=f7da249ee0ea4fe38db3f8ca11191225, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:48.971118 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:48.987564 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2851] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:49:48 2022] PUT /identity/v3/projects/f4c0c46c0d984a7fb3e10b0c1d8f2832/users/f7da249ee0ea4fe38db3f8ca11191225/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:49:48.998129 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:48.998409 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:48.998694 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:49.026754 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] MFA Rules not processed for user `f7da249ee0ea4fe38db3f8ca11191225`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:49:49.027757 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:49.038160 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:49.096394 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1429/2852] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:49:49.108174 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:49:49.110066 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:49:49.138286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:49.169374 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f0d1e-3aa0-488e-ad4f-1ae4101cdbab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:49:49.173318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:49:49.173969 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:49:49.174247 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:49:49.174543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:49:49.183447 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:49:49.184932 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050f94a-e601-46a5-a01f-1aa179267b1c service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:49:49.194183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2853] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:49:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:49:59.093549 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=22 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:49:59.189788 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=15 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:22.378941 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:22.498475 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5cf8fb-e5f8-42be-8645-f4772d39cccf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:22.504944 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:22.505830 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.506110 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.506415 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] PATH_INFO: `/v3/users/f7da249ee0ea4fe38db3f8ca11191225` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.507765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7da249ee0ea4fe38db3f8ca11191225)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:22.509775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:22.567030 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7da249ee0ea4fe38db3f8ca11191225'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:22.577091 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7da249ee0ea4fe38db3f8ca11191225'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:22.578052 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7da249ee0ea4fe38db3f8ca11191225'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:22.586733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7da249ee0ea4fe38db3f8ca11191225'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:22.587492 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7da249ee0ea4fe38db3f8ca11191225'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:22.611120 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1430/2854] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:50:22 2022] DELETE /identity/v3/users/f7da249ee0ea4fe38db3f8ca11191225 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:22.615892 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.616450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.616450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.673471 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:22.674689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:22.738181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2855] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:22.747869 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:22.749516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:22.794297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:22.857414 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c55a2d-e0d7-4d76-aa61-85b1424937e6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:22.862701 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:22.863304 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.863589 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.863864 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.874582 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:22.876062 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6d632-86b6-4ec9-bab8-8777c3a28fcd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:22.885966 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1431/2856] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:23.171089 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.172772 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:23.185855 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8938c3-cbb3-4d53-96bc-f2af109ee4f8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.192132 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.193206 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.193446 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.193755 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] PATH_INFO: `/v3/projects/f4c0c46c0d984a7fb3e10b0c1d8f2832` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.194981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c0c46c0d984a7fb3e10b0c1d8f2832)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.197013 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.248002 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c0c46c0d984a7fb3e10b0c1d8f2832', 'request_id': 'req-eca07e3d-96ca-45a2-bf6f-fd346170c067'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:23.253489 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c0c46c0d984a7fb3e10b0c1d8f2832', 'request_id': 'req-eca07e3d-96ca-45a2-bf6f-fd346170c067'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:23.261948 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c0c46c0d984a7fb3e10b0c1d8f2832', 'request_id': 'req-eca07e3d-96ca-45a2-bf6f-fd346170c067'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:23.273482 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] The token cache is being invalidate because project f4c0c46c0d984a7fb3e10b0c1d8f2832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:23.273759 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:23.274410 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eca07e3d-96ca-45a2-bf6f-fd346170c067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:23.275977 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2857] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:50:23 2022] DELETE /identity/v3/projects/f4c0c46c0d984a7fb3e10b0c1d8f2832 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:23.281072 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.281360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.281632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.336886 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:23.338147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:23.403117 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1432/2858] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:23.464988 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.465275 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.465542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.627097 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:23.628347 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:23.638136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2859] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:23.643337 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.653249 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f49114-a6d5-4a9b-aa35-cd4f3b5c5c06 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.658829 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.659487 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.659768 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.660054 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.660808 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:50:23.661112 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.662737 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.675611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1433/2860] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:50:23.680184 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.689824 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed4f1f7-27c4-4801-9320-5634c4a88f7e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.695762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.696463 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.696735 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.696962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.697658 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.699613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.729051 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2861] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:50:23 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:23.733427 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.744538 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7808b2-a5a7-4c2e-8a90-cad6b14f0250 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.806201 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.806863 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.807147 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.807420 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.808132 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.810047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.826345 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:23.833211 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1434/2862] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:50:23 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:23.837567 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.850452 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170b620-e6e4-4d45-8dd8-5673da6c6a16 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.858976 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.859656 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.859930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.860225 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.860995 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.862599 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.873860 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2863] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:23.878270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.889338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e062ab2-e9fd-4b8b-8315-7d46cb095c34 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.895196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.895866 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.896182 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.896454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.897198 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.898826 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189abc9e-0dea-4f14-bcc2-5b56c5844444 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.909777 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1435/2864] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:50:23.914075 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:23.924318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253885a1-d0f3-43e2-9d1b-ed893a628bf5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:23.930306 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:23.931412 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.931752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.932108 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] PATH_INFO: `/v3/projects/9cb9a8a3b54b4cdbb71b0f1044b1039e/users/790a9d0ed80c4ea2ad1c9fdf30af0e6f/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:23.949996 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb9a8a3b54b4cdbb71b0f1044b1039e, user_id=790a9d0ed80c4ea2ad1c9fdf30af0e6f, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:23.952896 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:23.955151 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:23.971095 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2865] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:50:23 2022] PUT /identity/v3/projects/9cb9a8a3b54b4cdbb71b0f1044b1039e/users/790a9d0ed80c4ea2ad1c9fdf30af0e6f/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:23.982145 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:23.982545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:23.983006 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:24.018576 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] MFA Rules not processed for user `790a9d0ed80c4ea2ad1c9fdf30af0e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:24.020464 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:24.095693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1436/2866] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:24.106702 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:24.108708 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:24.174713 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a1efc-a337-4c02-b166-757a492109f1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:24.182039 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03707ded8dc84180a4ea90986655b84e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:24.182989 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:24.183359 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:24.183720 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:24.194350 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:24.195926 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b1ef9-e9f7-498b-8480-bbb23de706f3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:24.205286 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2867] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:50:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:34.091458 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=38 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:34.201192 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:34.507223 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:34.590174 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d2ae2-ef5b-4cc7-9629-6d7f614bcd76 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:34.595167 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:34.596105 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:34.596387 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:34.596661 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] PATH_INFO: `/v3/users/790a9d0ed80c4ea2ad1c9fdf30af0e6f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:34.597878 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] RBAC: Authorizing `identity:delete_user(user_id=790a9d0ed80c4ea2ad1c9fdf30af0e6f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:34.599811 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:34.654327 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '790a9d0ed80c4ea2ad1c9fdf30af0e6f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:34.663918 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '790a9d0ed80c4ea2ad1c9fdf30af0e6f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:34.664605 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '790a9d0ed80c4ea2ad1c9fdf30af0e6f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:34.674629 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '790a9d0ed80c4ea2ad1c9fdf30af0e6f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:34.675992 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '790a9d0ed80c4ea2ad1c9fdf30af0e6f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:34.701503 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1437/2868] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:50:34 2022] DELETE /identity/v3/users/790a9d0ed80c4ea2ad1c9fdf30af0e6f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:34.706426 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:34.706933 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:34.707493 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:34.730925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:34.732177 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:34.809495 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2869] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:34.820447 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:34.822379 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:34.901530 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31b0632-6338-49ac-96b0-6b2c65ec387b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:34.905969 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:34.906945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:34.907224 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:34.907494 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:34.919047 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:34.920586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee68d09-e4a9-42db-b70a-1a49542e6666 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:34.932263 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1438/2870] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:50:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:35.222689 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.224270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:35.236469 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93fa42e-6d2e-4867-a543-e29194c650af None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.241293 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.242181 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.242458 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.242750 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] PATH_INFO: `/v3/projects/9cb9a8a3b54b4cdbb71b0f1044b1039e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.244148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb9a8a3b54b4cdbb71b0f1044b1039e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.246148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.294015 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb9a8a3b54b4cdbb71b0f1044b1039e', 'request_id': 'req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:35.299534 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb9a8a3b54b4cdbb71b0f1044b1039e', 'request_id': 'req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:35.308001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb9a8a3b54b4cdbb71b0f1044b1039e', 'request_id': 'req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:35.319207 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] The token cache is being invalidate because project 9cb9a8a3b54b4cdbb71b0f1044b1039e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:35.319487 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:35.320119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4cf2fa6-7dad-462b-9f98-a0c14b5828c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:35.321648 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2871] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:50:35 2022] DELETE /identity/v3/projects/9cb9a8a3b54b4cdbb71b0f1044b1039e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:35.327107 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.327593 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.327797 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.346196 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:35.347429 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:35.365965 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:35.414611 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1439/2872] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:35.422840 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.423119 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.423393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.442650 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:35.443887 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:35.453464 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2873] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:35.457997 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.468288 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211be31-7002-4d58-863c-91fb65bc43b0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.474385 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.475038 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.475312 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.475584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.476347 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:50:35.476656 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.478273 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.491026 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1440/2874] 10.222.0.55 () {58 vars in 1152 bytes} [Tue May 24 15:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:50:35.495213 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.504909 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f9a15-58b4-46b6-928c-7e6598fb07a5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.510814 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.511467 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.511738 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.512397 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.513074 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.515001 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.542805 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2875] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:50:35 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:35.547035 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.557343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4327df3-75a7-45ea-8be7-23dc869c5376 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.624153 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.625034 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.625299 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.625588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.627386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.630096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.655195 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1441/2876] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:50:35 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:35.659525 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.670316 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2d5fa-9320-49c2-af34-566ee4eeedd4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.676224 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.676894 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.677169 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.677437 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.678165 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.679780 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.691123 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2877] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:35.696801 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.707775 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb17858-7ca2-4736-848c-e3c85c6f74ca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.712946 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.713599 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.713879 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.714152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.714867 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.716526 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880bd71-56cd-4d2f-9621-dd6d12902feb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.727165 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1442/2878] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:50:35.731201 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.741480 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6a0e5-71be-4632-a950-b722a889f97d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.746333 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.747216 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.747490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.747752 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] PATH_INFO: `/v3/projects/6341a9bfc680413892cbfbb3e3845604/users/55472d5bd87849709ae105e8a49c62db/roles/491d70ccf49b4888bfe6d30f063be88e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.763107 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] RBAC: Authorizing `identity:create_grant(project_id=6341a9bfc680413892cbfbb3e3845604, user_id=55472d5bd87849709ae105e8a49c62db, role_id=491d70ccf49b4888bfe6d30f063be88e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.765774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.781462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2879] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:50:35 2022] PUT /identity/v3/projects/6341a9bfc680413892cbfbb3e3845604/users/55472d5bd87849709ae105e8a49c62db/roles/491d70ccf49b4888bfe6d30f063be88e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:35.786765 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac98a-13b0-4338-b0c7-3765350fd693 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.787057 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac98a-13b0-4338-b0c7-3765350fd693 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.787336 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aac98a-13b0-4338-b0c7-3765350fd693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.806496 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-71aac98a-13b0-4338-b0c7-3765350fd693 None None] MFA Rules not processed for user `108385be6e5b428a910af5b70d71b0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:35.807719 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71aac98a-13b0-4338-b0c7-3765350fd693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:35.875116 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1443/2880] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:35.885573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:35.887901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:35.923272 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:35.962583 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4d471-3664-4068-8d0d-5a06d346faba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:35.967202 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:35.967861 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:35.968172 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:35.968417 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:35.978226 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:35.979613 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:35.990150 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2881] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:39.947580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c597c-c632-426b-854d-a8b385945162 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:39.948189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c597c-c632-426b-854d-a8b385945162 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:39.948189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c597c-c632-426b-854d-a8b385945162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:39.977377 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-819c597c-c632-426b-854d-a8b385945162 None None] MFA Rules not processed for user `55472d5bd87849709ae105e8a49c62db`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:39.978521 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-819c597c-c632-426b-854d-a8b385945162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:39.997300 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:40.048919 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1444/2882] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:40.053341 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.063419 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08983887-e8c2-41b3-bd36-bde25e4c2528 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.069356 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.070339 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.070736 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.071090 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.071901 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.073930 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.108912 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2883] 10.222.0.55 () {60 vars in 1152 bytes} [Tue May 24 15:50:40 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:40.113003 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c597c-c632-426b-854d-a8b385945162 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.125023 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c597c-c632-426b-854d-a8b385945162 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.186200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.186976 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.187342 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.187683 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.188488 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.190746 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.212440 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1445/2884] 10.222.0.55 () {60 vars in 1143 bytes} [Tue May 24 15:50:40 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:40.216380 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.243809 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece3597b-c1d2-4338-af16-270578e26336 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.248569 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.249340 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.249798 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.250136 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.250928 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.253729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.267486 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2885] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:40.271581 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.281608 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1cd5e-ccf2-4593-8247-245140f51904 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.286569 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.287855 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.289088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.289088 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] PATH_INFO: `/v3/projects/962ac7d1acd7471d937731b813ec4123/users/d4362c7c43754ca6a636694d2349dab2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.306733 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=962ac7d1acd7471d937731b813ec4123, user_id=d4362c7c43754ca6a636694d2349dab2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.309457 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.324163 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1446/2886] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:50:40 2022] PUT /identity/v3/projects/962ac7d1acd7471d937731b813ec4123/users/d4362c7c43754ca6a636694d2349dab2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:40.328403 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.338796 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2765f95-47dd-40c0-abc3-541b46939334 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.400927 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.401603 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.401880 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.402152 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.402875 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.404514 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2456dbc5-8835-4435-ac3d-2b263329d2bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.415365 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2887] 10.222.0.55 () {58 vars in 1121 bytes} [Tue May 24 15:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:50:40.419632 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:40.431243 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f48143-0bef-446a-9f00-56a6dff2aef4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:40.436363 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:40.437297 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:40.437622 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:40.437876 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] PATH_INFO: `/v3/projects/962ac7d1acd7471d937731b813ec4123/users/d4362c7c43754ca6a636694d2349dab2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:40.440548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=962ac7d1acd7471d937731b813ec4123, user_id=d4362c7c43754ca6a636694d2349dab2, role_id=1d5097d60ffe4e02b13f5c0cc5227bc6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:40.443207 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:40.457631 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1447/2888] 10.222.0.55 () {60 vars in 1482 bytes} [Tue May 24 15:50:40 2022] PUT /identity/v3/projects/962ac7d1acd7471d937731b813ec4123/users/d4362c7c43754ca6a636694d2349dab2/roles/1d5097d60ffe4e02b13f5c0cc5227bc6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:44.804719 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7136e3-5466-42c8-86ce-0d5ec5780b9c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:44.805269 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7136e3-5466-42c8-86ce-0d5ec5780b9c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:44.805372 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7136e3-5466-42c8-86ce-0d5ec5780b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:44.832739 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9f7136e3-5466-42c8-86ce-0d5ec5780b9c None None] MFA Rules not processed for user `d4362c7c43754ca6a636694d2349dab2`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:44.833961 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9f7136e3-5466-42c8-86ce-0d5ec5780b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:44.905700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2889] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:44.922380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:44.926613 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:45.004718 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:45.011270 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8cdafb-324f-417d-82a9-d75d5bb97c05 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:45.018720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:45.019380 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:45.019658 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:45.019930 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:45.076905 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:45.078497 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:45.088073 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1448/2890] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:50:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:47.101671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:47.102148 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:47.102271 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:47.120837 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] MFA Rules not processed for user `55472d5bd87849709ae105e8a49c62db`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:47.122041 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:47.144640 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2891] 10.222.0.55 () {58 vars in 957 bytes} [Tue May 24 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:55.083884 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=27 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:57.115134 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:51:16.443833 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:51:16.445559 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:51:16.483178 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3ea0b-730e-499b-acfb-ca8def080a93 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:51:16.489541 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:51:16.490189 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:51:16.490486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:51:16.490741 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:51:16.492587 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:51:16.506386 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:51:16.508122 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:51:16.519123 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1449/2892] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:51:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:51:18.700962 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:51:18.702377 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:51:18.769361 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950bfb7-ee3e-4d8d-80ea-b2b89150a8a0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:51:18.773618 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:51:18.774265 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:51:18.774543 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:51:18.774810 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:51:18.784194 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:51:18.785628 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:51:18.795183 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2893] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:51:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:51:26.514584 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=15 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:51:28.791198 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=26 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:21.592251 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:21.593828 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:21.701194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d818767-e37a-42b8-b4c0-f6becc383184 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:21.707371 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:21.708042 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:21.708343 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:21.708612 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:21.710985 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:21.724301 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:21.726693 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:21.743540 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1450/2894] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:52:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:52:31.736295 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:53:13.239581 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:53:13.249706 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15776b-9a46-4076-8123-3183ac40a78c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:53:13.253686 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:53:13.254364 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:53:13.254644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:53:13.254917 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:53:13.256946 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:13.271453 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:53:13.273027 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:53:13.286846 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2895] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:53:14.475809 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:53:14.485652 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb7a99-50c1-45e4-97d5-34368a4131c9 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:53:14.489945 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:53:14.490586 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:53:14.490915 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:53:14.491221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:53:14.493222 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:14.507250 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:53:14.508725 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:53:14.520200 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1451/2896] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:53:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:53:23.280771 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:53:24.515558 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:53:26.613456 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:53:26.622843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a562f7a-07b5-4ffc-a3e4-6d8d926eaa25 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:53:26.626828 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:53:26.627478 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:53:26.627757 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:53:26.633024 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:53:26.635393 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:26.646143 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:53:26.648173 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:53:26.657571 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2897] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:53:26.770259 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:53:26.779539 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66685971-bb3f-4e8e-82c7-877b6ae0ba89 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:53:26.783580 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:53:26.784269 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:53:26.784548 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:53:26.784819 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:53:26.786720 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:26.800687 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:53:26.802215 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:53:26.814096 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1452/2898] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:53:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:53:36.653490 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:53:36.808411 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:16.911591 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.923871 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071394dd-0fed-42bf-8915-bb2c446e12ad service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.928630 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fbe062ad83f3490282626add7af4947a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.929276 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.929546 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.929815 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.931981 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:16.942819 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.944362 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.952462 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2899] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:26.949476 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:48.484192 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:48.498490 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092705b-7c88-455e-9a28-9bcda24d82ab service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:48.505517 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:48.506454 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:48.506973 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:48.507462 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:48.509883 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:48.524289 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:48.526338 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:48.539194 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1453/2900] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:54:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:50.775774 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:50.789019 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7392432-5dd7-4a86-81f9-fb0f85987893 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:50.794992 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c87b5039c28044988e7d8c418c286994', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:50.795638 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:50.795914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:50.796214 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:50.798079 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:50.811671 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:50.813331 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:50.824914 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2901] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:54:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:56.480888 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:56.482381 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:56.499362 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52eb94-3266-498d-97ce-a704d83454bb service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:56.505952 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:56.513847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:56.513847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:56.513847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:56.513847 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:56.534266 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:56.536152 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:56.550545 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1454/2902] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:54:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:59.746378 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:59.810573 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f02e0-364b-45f4-9b58-a7a8608d498b service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:59.815082 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:59.815963 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:59.816270 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:59.816542 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] PATH_INFO: `/v3/users/55472d5bd87849709ae105e8a49c62db` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:59.817729 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55472d5bd87849709ae105e8a49c62db)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:59.819697 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:59.872851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55472d5bd87849709ae105e8a49c62db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:59.881784 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55472d5bd87849709ae105e8a49c62db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:59.882379 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55472d5bd87849709ae105e8a49c62db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:59.883038 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55472d5bd87849709ae105e8a49c62db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:59.896318 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55472d5bd87849709ae105e8a49c62db'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:59.909411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2903] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:54:59 2022] DELETE /identity/v3/users/55472d5bd87849709ae105e8a49c62db => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:00.098460 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:00.110894 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189833f-3446-485d-93e2-8831993afb51 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:00.168804 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:00.169679 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:00.169950 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:00.170221 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] PATH_INFO: `/v3/projects/6341a9bfc680413892cbfbb3e3845604` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:00.171433 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6341a9bfc680413892cbfbb3e3845604)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:00.173360 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:00.224486 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6341a9bfc680413892cbfbb3e3845604', 'request_id': 'req-0bb3bc6c-55d5-4e83-b667-a67400e31a60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.230935 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6341a9bfc680413892cbfbb3e3845604', 'request_id': 'req-0bb3bc6c-55d5-4e83-b667-a67400e31a60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.241106 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6341a9bfc680413892cbfbb3e3845604', 'request_id': 'req-0bb3bc6c-55d5-4e83-b667-a67400e31a60'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.257588 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] The token cache is being invalidate because project 6341a9bfc680413892cbfbb3e3845604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:00.257872 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.259188 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.261998 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1455/2904] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:55:00 2022] DELETE /identity/v3/projects/6341a9bfc680413892cbfbb3e3845604 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:00.267044 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:00.268644 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:00.282936 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30581106-9f79-44c7-95c4-f97fc089b8e6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:00.341037 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:00.341925 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:00.342211 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:00.342484 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] PATH_INFO: `/v3/users/d4362c7c43754ca6a636694d2349dab2` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:00.343700 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4362c7c43754ca6a636694d2349dab2)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:00.345669 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:00.351330 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:00.402843 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4362c7c43754ca6a636694d2349dab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.411166 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4362c7c43754ca6a636694d2349dab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.411851 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4362c7c43754ca6a636694d2349dab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.412541 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4362c7c43754ca6a636694d2349dab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.425450 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4362c7c43754ca6a636694d2349dab2'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.445475 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2905] 10.222.0.55 () {58 vars in 1223 bytes} [Tue May 24 15:55:00 2022] DELETE /identity/v3/users/d4362c7c43754ca6a636694d2349dab2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:00.625726 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:00.627219 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:00.649951 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3bc6c-55d5-4e83-b667-a67400e31a60 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:00.718340 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '108385be6e5b428a910af5b70d71b0c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a64b5dceb3e44fc817c5b6566c89325', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:00.719161 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:00.719438 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:00.719712 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] PATH_INFO: `/v3/projects/962ac7d1acd7471d937731b813ec4123` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:00.720927 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=962ac7d1acd7471d937731b813ec4123)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:00.722868 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:00.729516 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:00.772634 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962ac7d1acd7471d937731b813ec4123', 'request_id': 'req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.778092 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962ac7d1acd7471d937731b813ec4123', 'request_id': 'req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.785843 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962ac7d1acd7471d937731b813ec4123', 'request_id': 'req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.797357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] The token cache is being invalidate because project 962ac7d1acd7471d937731b813ec4123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:00.797637 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.798357 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-210e1be4-b6b7-4926-a3e0-57f2679fd8ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:00.799979 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1456/2906] 10.222.0.55 () {58 vars in 1232 bytes} [Tue May 24 15:55:00 2022] DELETE /identity/v3/projects/962ac7d1acd7471d937731b813ec4123 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:10.439137 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:10.794422 dv-cinder-843153-1-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:25.550163 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.551762 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:25.626858 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4719986-2121-4b42-bdd0-9e5eb05856ec admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.688749 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e5a44c30a0b430f90cabf9b3084ec28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c13cc48481ea47fca3a3ef6370ade58e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.689411 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.689690 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.689965 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.691804 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:25.703621 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.705235 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb0c75-c614-4840-8d85-c23a805d7584 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.715242 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2907] 10.222.0.55 () {60 vars in 1365 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:35.710991 dv-cinder-843153-1-iscsi devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=58 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}